[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:19:34 dist-test-slave-4wm5 krb5kdc[3407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:19:34 dist-test-slave-4wm5 krb5kdc[3407](info): set up 2 sockets
Sep 25 09:19:34 dist-test-slave-4wm5 krb5kdc[3407](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 09:19:35.058415 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:19:35 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:19:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:19:35.255580 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45441
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:35.590701 3420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:35.591022 3420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:35.591111 3420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:35.598623 3420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:19:35.598750 3420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:35.598799 3420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:35.598850 3420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:19:35.598896 3420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:19:35.609236 3420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:35.610627 3420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:35.611963 3420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:35.625168 3425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:35.625305 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:35.625329 3426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:35.625669 3420 server_base.cc:1034] running on GCE node
I20260925 09:19:35.626833 3420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:35.628443 3420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:35.629693 3420 hybrid_clock.cc:648] HybridClock initialized: now 1790327975629654 us; error 53 us; skew 500 ppm
I20260925 09:19:35.630154 3420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:35 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327975, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:35.633654 3420 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:19:35.635313 3420 webserver.cc:460] Webserver started at http://127.3.82.62:33363/ using document root <none> and password file <none>
I20260925 09:19:35.636052 3420 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:35.636159 3420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:35.636516 3420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:35.639761 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "9103918ead0349d3bde9d17d72604c69"
format_stamp: "Formatted at 2026-09-25 09:19:35 on dist-test-slave-4wm5"
server_key: "0767752beaa3e21b2f242e6812bbf079"
server_key_iv: "e1d69cae0ed882b5e1b72ff5e2de7f4c"
server_key_version: "encryptionkey@0"
I20260925 09:19:35.640588 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "9103918ead0349d3bde9d17d72604c69"
format_stamp: "Formatted at 2026-09-25 09:19:35 on dist-test-slave-4wm5"
server_key: "0767752beaa3e21b2f242e6812bbf079"
server_key_iv: "e1d69cae0ed882b5e1b72ff5e2de7f4c"
server_key_version: "encryptionkey@0"
I20260925 09:19:35.646240 3420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:19:35.650337 3435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:35.651847 3420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:19:35.652064 3420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "9103918ead0349d3bde9d17d72604c69"
format_stamp: "Formatted at 2026-09-25 09:19:35 on dist-test-slave-4wm5"
server_key: "0767752beaa3e21b2f242e6812bbf079"
server_key_iv: "e1d69cae0ed882b5e1b72ff5e2de7f4c"
server_key_version: "encryptionkey@0"
I20260925 09:19:35.652292 3420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:35.669220 3420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:35.673748 3420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:35.674119 3420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:35.694038 3420 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45441
I20260925 09:19:35.694039 3486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45441 every 8 connection(s)
I20260925 09:19:35.695536 3420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:19:35.699502 3487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:35.704885 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 3420
I20260925 09:19:35.705255 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:19:35.707872 3400 external_mini_cluster.cc:1428] Setting key 2d4d5f01c089c831050e04423891da53
I20260925 09:19:35.713033 3487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69: Bootstrap starting.
I20260925 09:19:35.717396 3487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:35.718864 3487 log.cc:824] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:35.722417 3487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69: No bootstrap required, opened a new log
I20260925 09:19:35.732111 3487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9103918ead0349d3bde9d17d72604c69" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45441 } }
I20260925 09:19:35.732662 3487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:35.732781 3487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9103918ead0349d3bde9d17d72604c69, State: Initialized, Role: FOLLOWER
I20260925 09:19:35.733506 3487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9103918ead0349d3bde9d17d72604c69" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45441 } }
I20260925 09:19:35.733788 3487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:19:35.733952 3487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:19:35.734130 3487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:35.736438 3487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9103918ead0349d3bde9d17d72604c69" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45441 } }
I20260925 09:19:35.737053 3487 leader_election.cc:302] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9103918ead0349d3bde9d17d72604c69; no voters:
I20260925 09:19:35.737768 3487 leader_election.cc:288] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:19:35.737994 3492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:35.739478 3492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [term 1 LEADER]: Becoming Leader. State: Replica: 9103918ead0349d3bde9d17d72604c69, State: Running, Role: LEADER
I20260925 09:19:35.740162 3492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9103918ead0349d3bde9d17d72604c69" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45441 } }
I20260925 09:19:35.740396 3487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:19:35.742946 3493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9103918ead0349d3bde9d17d72604c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9103918ead0349d3bde9d17d72604c69" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45441 } } }
I20260925 09:19:35.743161 3494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9103918ead0349d3bde9d17d72604c69. Latest consensus state: current_term: 1 leader_uuid: "9103918ead0349d3bde9d17d72604c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9103918ead0349d3bde9d17d72604c69" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45441 } } }
I20260925 09:19:35.743417 3494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:35.743420 3493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69 [sys.catalog]: This master's current role is: LEADER
Sep 25 09:19:35 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:35.756608 3497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:19:35.762579 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:35.712947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.3.82.62:45441)
0925 09:19:35.714044 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:35.714064 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:35.715508 (+ 1444us) server_negotiation.cc:407] Connection header received
0925 09:19:35.727552 (+ 12044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:35.727587 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:35.728263 (+ 676us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:35.728818 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:35.732345 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:35.734450 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:35.736712 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:35.737902 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:35.738134 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:35.748195 (+ 10061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:35.748214 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:35.748247 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:35.748344 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:35.754314 (+ 5970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:35.756057 (+ 1743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:35.756072 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:35.756085 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:35.756429 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:35.757139 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:35.757144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:35.757148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:35.758035 (+ 887us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:35.758378 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:35.759844 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0925 09:19:35.760542 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":446,"threads_started":1}
I20260925 09:19:35.763854 3497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:19:35.767297 3509 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:19:35.775749 3497 catalog_manager.cc:1347] Generated new cluster ID: 8572c0d1d36c4ac7bbd69cac40b259b4
I20260925 09:19:35.775918 3497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:19:35.818744 3497 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:19:35.820724 3497 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:19:35.829993 3497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9103918ead0349d3bde9d17d72604c69: Generated new TSK 0
I20260925 09:19:35.831090 3497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:19:35.895092 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45441
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:36.252673 3513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:36.253032 3513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:36.253146 3513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:36.262111 3513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:36.262247 3513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:36.262379 3513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:19:36.274187 3513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:36.275656 3513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:36.277355 3513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:36.292683 3518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:36.292609 3521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:36.292734 3519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:36.293844 3513 server_base.cc:1034] running on GCE node
I20260925 09:19:36.294610 3513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:36.295723 3513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:36.297000 3513 hybrid_clock.cc:648] HybridClock initialized: now 1790327976296933 us; error 65 us; skew 500 ppm
I20260925 09:19:36.297433 3513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:36 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327976, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:36.301095 3513 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:19:36.302802 3513 webserver.cc:460] Webserver started at http://127.3.82.1:44145/ using document root <none> and password file <none>
I20260925 09:19:36.303529 3513 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:36.303633 3513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:36.303906 3513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:36.307197 3513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "d13df51b98824e4dbc608f489721984b"
format_stamp: "Formatted at 2026-09-25 09:19:36 on dist-test-slave-4wm5"
server_key: "a189aef7b9d0f611f8da3fdb4c5b2bad"
server_key_iv: "d035b0adb5419f5a603da41adc7a74d6"
server_key_version: "encryptionkey@0"
I20260925 09:19:36.308022 3513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "d13df51b98824e4dbc608f489721984b"
format_stamp: "Formatted at 2026-09-25 09:19:36 on dist-test-slave-4wm5"
server_key: "a189aef7b9d0f611f8da3fdb4c5b2bad"
server_key_iv: "d035b0adb5419f5a603da41adc7a74d6"
server_key_version: "encryptionkey@0"
I20260925 09:19:36.314076 3513 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:19:36.318310 3528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:36.320096 3513 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:19:36.320307 3513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "d13df51b98824e4dbc608f489721984b"
format_stamp: "Formatted at 2026-09-25 09:19:36 on dist-test-slave-4wm5"
server_key: "a189aef7b9d0f611f8da3fdb4c5b2bad"
server_key_iv: "d035b0adb5419f5a603da41adc7a74d6"
server_key_version: "encryptionkey@0"
I20260925 09:19:36.320531 3513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:36.366521 3513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:36.371402 3513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:36.371728 3513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:36.372828 3513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:36.374389 3513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:36.374472 3513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:36.374607 3513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:36.374670 3513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:36.413666 3513 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:44991
I20260925 09:19:36.413712 3640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:44991 every 8 connection(s)
I20260925 09:19:36.415150 3513 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:19:36.425076 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 3513
I20260925 09:19:36.425323 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:19:36.426035 3400 external_mini_cluster.cc:1428] Setting key 8ba384dd93fadc3bd2f015f166710187
Sep 25 09:19:36 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327976, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:36.441743 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:36.418105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:37709 (local address 127.3.82.62:45441)
0925 09:19:36.418594 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:36.418599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:36.419847 (+ 1248us) server_negotiation.cc:407] Connection header received
0925 09:19:36.422153 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:36.422159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:36.422258 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:36.422397 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:36.424051 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.425019 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.427322 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.428012 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.428151 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:36.433250 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:36.433257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:36.433260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:36.433349 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:36.437397 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:36.438593 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:36.438602 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:36.438609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:36.438787 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:36.439321 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:36.439329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:36.439336 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:36.439757 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:36.439923 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:36.440923 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0925 09:19:36.441220 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":245,"threads_started":1}
I20260925 09:19:36.442675 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:36.418848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45441 (local address 127.3.82.1:37709)
0925 09:19:36.419663 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:36.419704 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:36.421748 (+ 2044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:36.422681 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:36.422691 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:36.423236 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:36.423860 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:36.423878 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.425249 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.425255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:36.427167 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:36.427183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.428372 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.428382 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:36.428716 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:36.429548 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:36.429628 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:36.432925 (+ 3297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:36.437625 (+ 4700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:36.437635 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:36.437652 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:36.438379 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:36.438931 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:36.438936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:36.438940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:36.439177 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:36.439892 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:36.439902 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:36.440378 (+ 476us) client_negotiation.cc:769] Sending connection context
0925 09:19:36.440737 (+ 359us) client_negotiation.cc:240] Negotiation successful
0925 09:19:36.441256 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":289,"threads_started":1}
I20260925 09:19:36.444521 3641 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45441
I20260925 09:19:36.444940 3641 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:19:36.446039 3641 heartbeater.cc:511] Master 127.3.82.62:45441 requested a full tablet report, sending...
I20260925 09:19:36.448659 3452 ts_manager.cc:194] Registered new tserver with Master: d13df51b98824e4dbc608f489721984b (127.3.82.1:44991)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:19:36.450968 3452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:37709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:19:36.474551 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45441
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:36.811328 3647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:36.811627 3647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:36.811790 3647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:36.819234 3647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:36.819370 3647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:36.819597 3647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:19:36.830478 3647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:36.832010 3647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:36.833441 3647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:36.848135 3655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:36.848239 3653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:36.848219 3652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:36.851771 3647 server_base.cc:1034] running on GCE node
I20260925 09:19:36.852560 3647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:36.853720 3647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:36.855000 3647 hybrid_clock.cc:648] HybridClock initialized: now 1790327976854918 us; error 88 us; skew 500 ppm
I20260925 09:19:36.855384 3647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:36 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327976, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:36.859350 3647 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:19:36.861085 3647 webserver.cc:460] Webserver started at http://127.3.82.2:46343/ using document root <none> and password file <none>
I20260925 09:19:36.861956 3647 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:36.862105 3647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:36.862434 3647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:36.865492 3647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "2d94227262ba4ecdb12effbc7f17fdba"
format_stamp: "Formatted at 2026-09-25 09:19:36 on dist-test-slave-4wm5"
server_key: "12f8c225c3f97a67bf86d5bac6391ec5"
server_key_iv: "67561911137142e7d3c656d94e367be7"
server_key_version: "encryptionkey@0"
I20260925 09:19:36.866400 3647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "2d94227262ba4ecdb12effbc7f17fdba"
format_stamp: "Formatted at 2026-09-25 09:19:36 on dist-test-slave-4wm5"
server_key: "12f8c225c3f97a67bf86d5bac6391ec5"
server_key_iv: "67561911137142e7d3c656d94e367be7"
server_key_version: "encryptionkey@0"
I20260925 09:19:36.872192 3647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:19:36.876019 3662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:36.877570 3647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:19:36.877774 3647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "2d94227262ba4ecdb12effbc7f17fdba"
format_stamp: "Formatted at 2026-09-25 09:19:36 on dist-test-slave-4wm5"
server_key: "12f8c225c3f97a67bf86d5bac6391ec5"
server_key_iv: "67561911137142e7d3c656d94e367be7"
server_key_version: "encryptionkey@0"
I20260925 09:19:36.878067 3647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:36.908015 3647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:36.912890 3647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:36.913204 3647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:36.914316 3647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:36.915766 3647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:36.915869 3647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:36.916014 3647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:36.916083 3647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:36.952590 3647 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:38495
I20260925 09:19:36.952646 3774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:38495 every 8 connection(s)
I20260925 09:19:36.954000 3647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:19:36.964190 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 3647
I20260925 09:19:36.964401 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:19:36.965001 3400 external_mini_cluster.cc:1428] Setting key 38d2e80fe9d3504d95acff90ec1334ef
Sep 25 09:19:36 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327976, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:36.977924 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:36.956731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:43695 (local address 127.3.82.62:45441)
0925 09:19:36.957352 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:36.957356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:36.958545 (+ 1189us) server_negotiation.cc:407] Connection header received
0925 09:19:36.960502 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:36.960506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:36.960591 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:36.960698 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:36.962460 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.963284 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.966013 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.966522 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.966684 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:36.970708 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:36.970720 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:36.970727 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:36.970781 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:36.974103 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:36.975267 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:36.975271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:36.975274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:36.975426 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:36.975969 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:36.975972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:36.975974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:36.976260 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:36.976374 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:36.977375 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0925 09:19:36.977565 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":278,"threads_started":1}
I20260925 09:19:36.979674 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:36.957403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45441 (local address 127.3.82.2:43695)
0925 09:19:36.958331 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:36.958386 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:36.960202 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:36.960966 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:36.960978 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:36.961554 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:36.962259 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:36.962278 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.963476 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.963486 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:36.965717 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:36.965739 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:36.966783 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:36.966792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:36.967125 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:36.967929 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:36.967953 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:36.970385 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:36.974303 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:36.974318 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:36.974343 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:36.975107 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:36.975558 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:36.975566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:36.975572 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:36.975850 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:36.976393 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:36.976405 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:36.976936 (+ 531us) client_negotiation.cc:769] Sending connection context
0925 09:19:36.977267 (+ 331us) client_negotiation.cc:240] Negotiation successful
0925 09:19:36.977987 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":407,"threads_started":1}
I20260925 09:19:36.981354 3775 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45441
I20260925 09:19:36.981714 3775 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:36.982649 3775 heartbeater.cc:511] Master 127.3.82.62:45441 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:19:36.984345 3452 ts_manager.cc:194] Registered new tserver with Master: 2d94227262ba4ecdb12effbc7f17fdba (127.3.82.2:38495)
I20260925 09:19:36.985256 3452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:43695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:19:37.014932 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45441
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:37.361218 3781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:37.361577 3781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:37.361759 3781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:37.369539 3781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:37.369716 3781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:37.369976 3781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:19:37.380532 3781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:37.382052 3781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:37.383499 3781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:37.397677 3786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:37.397488 3787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:37.397846 3789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:37.399250 3781 server_base.cc:1034] running on GCE node
I20260925 09:19:37.400043 3781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:37.401135 3781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:37.402432 3781 hybrid_clock.cc:648] HybridClock initialized: now 1790327977402350 us; error 49 us; skew 500 ppm
I20260925 09:19:37.402851 3781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:37.406364 3781 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:19:37.408120 3781 webserver.cc:460] Webserver started at http://127.3.82.3:46579/ using document root <none> and password file <none>
I20260925 09:19:37.409001 3781 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:37.409159 3781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:37.409515 3781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:37.412931 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "83f9b4855b5b4b8a9b62decc7dd199f2"
format_stamp: "Formatted at 2026-09-25 09:19:37 on dist-test-slave-4wm5"
server_key: "4bd0fb99c49c4c2c8bc548f3ad7dbc34"
server_key_iv: "9190ed940ed429a7601f82d576130039"
server_key_version: "encryptionkey@0"
I20260925 09:19:37.414094 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "83f9b4855b5b4b8a9b62decc7dd199f2"
format_stamp: "Formatted at 2026-09-25 09:19:37 on dist-test-slave-4wm5"
server_key: "4bd0fb99c49c4c2c8bc548f3ad7dbc34"
server_key_iv: "9190ed940ed429a7601f82d576130039"
server_key_version: "encryptionkey@0"
I20260925 09:19:37.421557 3781 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260925 09:19:37.425755 3796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:37.427441 3781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:19:37.427651 3781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "83f9b4855b5b4b8a9b62decc7dd199f2"
format_stamp: "Formatted at 2026-09-25 09:19:37 on dist-test-slave-4wm5"
server_key: "4bd0fb99c49c4c2c8bc548f3ad7dbc34"
server_key_iv: "9190ed940ed429a7601f82d576130039"
server_key_version: "encryptionkey@0"
I20260925 09:19:37.427891 3781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:37.451740 3781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:37.455346 3641 heartbeater.cc:503] Master 127.3.82.62:45441 was elected leader, sending a full tablet report...
I20260925 09:19:37.456987 3781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:37.457370 3781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:37.458555 3781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:37.459966 3781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:37.460076 3781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:37.460240 3781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:37.460311 3781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:37.496119 3781 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:37795
I20260925 09:19:37.496176 3908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:37795 every 8 connection(s)
I20260925 09:19:37.497541 3781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:19:37.499253 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 3781
I20260925 09:19:37.499445 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:19:37.499886 3400 external_mini_cluster.cc:1428] Setting key 61fad1b3eeb66606a1ef62d98757961e
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:37.521136 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.500480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:58503 (local address 127.3.82.62:45441)
0925 09:19:37.501105 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:37.501111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:37.502553 (+ 1442us) server_negotiation.cc:407] Connection header received
0925 09:19:37.504983 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:37.504988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:37.505091 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:37.505240 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:37.507032 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.507907 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.509651 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.510204 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.510356 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.514325 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:37.514330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:37.514334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:37.514371 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:37.517489 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.518447 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.518452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.518455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.518589 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.519086 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.519097 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.519099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.519378 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:37.519553 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:37.520366 (+ 813us) server_negotiation.cc:299] Negotiation successful
0925 09:19:37.520691 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":350,"threads_started":1}
I20260925 09:19:37.522388 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.501256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45441 (local address 127.3.82.3:58503)
0925 09:19:37.502348 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:37.502391 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:37.504686 (+ 2295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:37.505513 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:37.505523 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:37.506204 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:37.506845 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:37.506865 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.508083 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.508097 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:37.509478 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:37.509491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.510470 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.510483 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:37.510743 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.511520 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:37.511545 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:37.514075 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:37.517662 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:37.517673 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:37.517695 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:37.518291 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:37.518717 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:37.518723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:37.518727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:37.518956 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:37.519538 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:37.519547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:37.519952 (+ 405us) client_negotiation.cc:769] Sending connection context
0925 09:19:37.520255 (+ 303us) client_negotiation.cc:240] Negotiation successful
0925 09:19:37.520950 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":499,"threads_started":1}
I20260925 09:19:37.524170 3909 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45441
I20260925 09:19:37.524539 3909 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:37.525377 3909 heartbeater.cc:511] Master 127.3.82.62:45441 requested a full tablet report, sending...
I20260925 09:19:37.527134 3452 ts_manager.cc:194] Registered new tserver with Master: 83f9b4855b5b4b8a9b62decc7dd199f2 (127.3.82.3:37795)
I20260925 09:19:37.528286 3452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:58503
I20260925 09:19:37.529706 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:37.580780 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.562855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.3.82.1:44991)
0925 09:19:37.563577 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:37.563584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:37.563615 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:19:37.563823 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:37.563832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:37.564220 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:37.564379 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:37.566298 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.567487 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.568958 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.569630 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.569743 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.574462 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:37.574479 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:37.574497 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:37.574547 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:37.577702 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.578489 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.578497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.578499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.578647 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.579044 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.579048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.579050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.579435 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:37.579679 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:37.580147 (+ 468us) server_negotiation.cc:299] Negotiation successful
0925 09:19:37.580454 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":385,"threads_started":1}
W20260925 09:19:37.582267 3556 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35550
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:37.628890 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.613999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.3.82.1:44991)
0925 09:19:37.614244 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:37.614252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:37.614500 (+ 248us) server_negotiation.cc:407] Connection header received
0925 09:19:37.614756 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:37.614761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:37.614846 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:37.614978 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:37.616475 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.617233 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.618708 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.619202 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.619363 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.623682 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:37.623687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:37.623690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:37.623729 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:37.626684 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.627340 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.627344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.627346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.627474 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.627804 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.627807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.627809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.628070 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:37.628239 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:37.628477 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 09:19:37.628643 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:19:37.630712 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 3513
I20260925 09:19:37.644973 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.631686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35566 (local address 127.3.82.1:44991)
0925 09:19:37.631861 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:37.631866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:37.632348 (+ 482us) server_negotiation.cc:407] Connection header received
0925 09:19:37.632480 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:37.632483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:37.632541 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:37.632676 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:37.633897 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.634643 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.636172 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.636583 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.636689 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.639326 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:37.639331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:37.639334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:37.639371 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:37.642252 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.643002 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.643007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.643009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.643147 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.643476 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.643479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.643481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.643763 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:37.643937 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:37.644492 (+ 555us) server_negotiation.cc:299] Negotiation successful
0925 09:19:37.644687 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:19:37.645784 3636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:37.734076 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 3513
I20260925 09:19:37.751049 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 3647
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:19:37.769601 3924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.752066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54510 (local address 127.3.82.2:38495)
0925 09:19:37.752666 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:37.752673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:37.752700 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:19:37.752763 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:37.752770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:37.753027 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:37.753170 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:37.754536 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.755657 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.757108 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.757862 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.758032 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.762815 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:37.762834 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:37.762851 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:37.762899 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:37.766272 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.767180 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.767189 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.767192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.767332 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.767941 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.767945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.767947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.768409 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:37.768672 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:37.769002 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 09:19:37.769279 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":296,"threads_started":1}
I20260925 09:19:37.771181 3770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:37.859762 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 3647
I20260925 09:19:37.875694 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 3781
Sep 25 09:19:37 dist-test-slave-4wm5 krb5kdc[3407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:19:37.893380 3926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:37.876738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.3.82.3:37795)
0925 09:19:37.877449 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:37.877456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:37.877485 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:19:37.877546 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:37.877552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:37.877782 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:37.878000 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:37.879245 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.880270 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.881688 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:37.882363 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:37.882511 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:37.887003 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:37.887020 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:37.887037 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:37.887084 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:37.890312 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.891241 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.891248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.891251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.891383 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:37.891803 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:37.891806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:37.891808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:37.892163 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:37.892388 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:37.892746 (+ 358us) server_negotiation.cc:299] Negotiation successful
0925 09:19:37.892955 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":398,"threads_started":1}
I20260925 09:19:37.895141 3904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:37.982798 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 3781
I20260925 09:19:37.999217 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 3420
I20260925 09:19:38.000572 3482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:38.109088 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 3420
2026-09-25T09:19:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4171 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:19:38 dist-test-slave-4wm5 krb5kdc[3931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:19:38 dist-test-slave-4wm5 krb5kdc[3931](info): set up 2 sockets
Sep 25 09:19:38 dist-test-slave-4wm5 krb5kdc[3931](info): commencing operation
krb5kdc: starting...
W20260925 09:19:39.218467 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:19:39 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:19:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:19:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:19:39.419042 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36275
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:39.771530 3944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:39.771986 3944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:39.772171 3944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:39.780915 3944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:19:39.781163 3944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:39.781288 3944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:39.781399 3944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:19:39.781495 3944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:19:39.792409 3944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:39.794023 3944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:39.795533 3944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:39.808331 3949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:39.809109 3952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:39.809175 3950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:39.810034 3944 server_base.cc:1034] running on GCE node
I20260925 09:19:39.811049 3944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:39.812759 3944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:39.814002 3944 hybrid_clock.cc:648] HybridClock initialized: now 1790327979813930 us; error 68 us; skew 500 ppm
I20260925 09:19:39.814415 3944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:39 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327979, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:39.822800 3944 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:19:39.824759 3944 webserver.cc:460] Webserver started at http://127.3.82.62:41307/ using document root <none> and password file <none>
I20260925 09:19:39.825560 3944 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:39.825730 3944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:39.826117 3944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:39.829285 3944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "01c509f5450e43d6938ad5debc6a2b29"
format_stamp: "Formatted at 2026-09-25 09:19:39 on dist-test-slave-4wm5"
server_key: "f41ba004feeaf29a75472257e6e0c77a"
server_key_iv: "d7a09e497c699fcfd93e8bb727e2f86a"
server_key_version: "encryptionkey@0"
I20260925 09:19:39.830199 3944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "01c509f5450e43d6938ad5debc6a2b29"
format_stamp: "Formatted at 2026-09-25 09:19:39 on dist-test-slave-4wm5"
server_key: "f41ba004feeaf29a75472257e6e0c77a"
server_key_iv: "d7a09e497c699fcfd93e8bb727e2f86a"
server_key_version: "encryptionkey@0"
I20260925 09:19:39.835850 3944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:19:39.840106 3959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:39.841732 3944 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 09:19:39.841990 3944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "01c509f5450e43d6938ad5debc6a2b29"
format_stamp: "Formatted at 2026-09-25 09:19:39 on dist-test-slave-4wm5"
server_key: "f41ba004feeaf29a75472257e6e0c77a"
server_key_iv: "d7a09e497c699fcfd93e8bb727e2f86a"
server_key_version: "encryptionkey@0"
I20260925 09:19:39.842271 3944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:39.870505 3944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:39.875360 3944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:39.875696 3944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:39.899492 3944 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36275
I20260925 09:19:39.899559 4010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36275 every 8 connection(s)
I20260925 09:19:39.901094 3944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:19:39.905280 4011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:39.910727 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 3944
I20260925 09:19:39.910972 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:19:39.911648 3400 external_mini_cluster.cc:1428] Setting key de318a2ed4c0d8b05f6d087dcccaed50
I20260925 09:19:39.918133 4011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29: Bootstrap starting.
I20260925 09:19:39.921620 4011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:39.923162 4011 log.cc:824] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:19:39 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:39.927659 4011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29: No bootstrap required, opened a new log
I20260925 09:19:39.936214 4011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c509f5450e43d6938ad5debc6a2b29" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36275 } }
I20260925 09:19:39.936645 4011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:39.936666 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:39.913998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.3.82.62:36275)
0925 09:19:39.914869 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:39.914882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:39.914927 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:19:39.916801 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:39.916829 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:39.917292 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:39.917747 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:39.919318 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:39.920738 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:39.922192 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:39.922955 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:39.923190 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:39.928433 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:39.928446 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:39.928495 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:39.928541 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:39.932080 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:39.932919 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:39.932927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:39.932934 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:39.933133 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:39.933540 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:39.933544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:39.933546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:39.934219 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:39.934562 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:39.935081 (+ 519us) server_negotiation.cc:299] Negotiation successful
0925 09:19:39.935540 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":364,"threads_started":1}
I20260925 09:19:39.936812 4011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c509f5450e43d6938ad5debc6a2b29, State: Initialized, Role: FOLLOWER
I20260925 09:19:39.937625 4011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c509f5450e43d6938ad5debc6a2b29" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36275 } }
I20260925 09:19:39.937994 4011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:19:39.938169 4011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:19:39.938381 4011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:39.941125 4011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c509f5450e43d6938ad5debc6a2b29" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36275 } }
I20260925 09:19:39.941733 4011 leader_election.cc:302] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01c509f5450e43d6938ad5debc6a2b29; no voters:
I20260925 09:19:39.942433 4011 leader_election.cc:288] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:19:39.942536 4016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:39.943795 4016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [term 1 LEADER]: Becoming Leader. State: Replica: 01c509f5450e43d6938ad5debc6a2b29, State: Running, Role: LEADER
I20260925 09:19:39.944413 4016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c509f5450e43d6938ad5debc6a2b29" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36275 } }
I20260925 09:19:39.944695 4011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:19:39.947750 4017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01c509f5450e43d6938ad5debc6a2b29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c509f5450e43d6938ad5debc6a2b29" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36275 } } }
I20260925 09:19:39.947840 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01c509f5450e43d6938ad5debc6a2b29. Latest consensus state: current_term: 1 leader_uuid: "01c509f5450e43d6938ad5debc6a2b29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c509f5450e43d6938ad5debc6a2b29" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36275 } } }
I20260925 09:19:39.949185 4017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:39.949204 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:39.952875 4023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:19:39.958801 4023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:19:39.968289 4023 catalog_manager.cc:1347] Generated new cluster ID: ed9cbbb6154c45e1826f4348f8f8763e
I20260925 09:19:39.968497 4023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:19:40.004886 4023 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:19:40.006510 4023 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:19:40.024799 4023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01c509f5450e43d6938ad5debc6a2b29: Generated new TSK 0
I20260925 09:19:40.026106 4023 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:19:40.080322 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36275
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:40.429711 4037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:40.430086 4037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:40.430228 4037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:40.437968 4037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:40.438088 4037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:40.438294 4037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:19:40.450893 4037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:40.452353 4037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:40.453887 4037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:40.469182 4042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:40.469270 4045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:40.469296 4043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:40.470534 4037 server_base.cc:1034] running on GCE node
I20260925 09:19:40.471385 4037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:40.472545 4037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:40.474007 4037 hybrid_clock.cc:648] HybridClock initialized: now 1790327980473788 us; error 42 us; skew 500 ppm
I20260925 09:19:40.474431 4037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:40 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327980, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:40.478458 4037 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:19:40.480494 4037 webserver.cc:460] Webserver started at http://127.3.82.1:45487/ using document root <none> and password file <none>
I20260925 09:19:40.481338 4037 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:40.481510 4037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:40.481931 4037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:40.485277 4037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "ee737e16933d49f2aa9136ab8ded1e88"
format_stamp: "Formatted at 2026-09-25 09:19:40 on dist-test-slave-4wm5"
server_key: "fc0b039cce09da35becf1343420c498d"
server_key_iv: "34cfc46f0a87e56bba862b50a8f3dd0c"
server_key_version: "encryptionkey@0"
I20260925 09:19:40.486284 4037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "ee737e16933d49f2aa9136ab8ded1e88"
format_stamp: "Formatted at 2026-09-25 09:19:40 on dist-test-slave-4wm5"
server_key: "fc0b039cce09da35becf1343420c498d"
server_key_iv: "34cfc46f0a87e56bba862b50a8f3dd0c"
server_key_version: "encryptionkey@0"
I20260925 09:19:40.492556 4037 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:19:40.496675 4052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:40.498407 4037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:19:40.498632 4037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "ee737e16933d49f2aa9136ab8ded1e88"
format_stamp: "Formatted at 2026-09-25 09:19:40 on dist-test-slave-4wm5"
server_key: "fc0b039cce09da35becf1343420c498d"
server_key_iv: "34cfc46f0a87e56bba862b50a8f3dd0c"
server_key_version: "encryptionkey@0"
I20260925 09:19:40.498883 4037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:40.516966 4037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:40.522305 4037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:40.522641 4037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:40.523811 4037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:40.525380 4037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:40.525496 4037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:40.525691 4037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:40.525774 4037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:40.564239 4037 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:45389
I20260925 09:19:40.564280 4164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:45389 every 8 connection(s)
I20260925 09:19:40.565755 4037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:19:40.571321 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4037
I20260925 09:19:40.571556 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:19:40.572340 3400 external_mini_cluster.cc:1428] Setting key d62129b6e423f01f94e53969682663a7
Sep 25 09:19:40 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327980, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:40.590983 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:40.568865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:41943 (local address 127.3.82.62:36275)
0925 09:19:40.569376 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:40.569381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:40.570459 (+ 1078us) server_negotiation.cc:407] Connection header received
0925 09:19:40.573252 (+ 2793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:40.573257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:40.573357 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:40.573483 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:40.575415 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:40.576226 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:40.578876 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:40.579404 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:40.579646 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:40.584126 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:40.584132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:40.584136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:40.584171 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:40.586990 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:40.588226 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:40.588230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:40.588232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:40.588350 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:40.589052 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:40.589055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:40.589057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:40.589331 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:40.589444 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:40.590532 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0925 09:19:40.590719 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":241,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:19:40.592777 4168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:40.569455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36275 (local address 127.3.82.1:41943)
0925 09:19:40.570264 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:40.570312 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:40.572902 (+ 2590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:40.573756 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:40.573766 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:40.574372 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:40.575208 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:40.575238 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:40.576413 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:40.576422 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:40.578696 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:40.578716 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:40.579735 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:40.579743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:40.580090 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:40.580993 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:40.581017 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:40.583818 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:40.587177 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:40.587192 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:40.587225 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:40.588081 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:40.588471 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:40.588480 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:40.588486 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:40.588900 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:40.589473 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:40.589482 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:40.590051 (+ 569us) client_negotiation.cc:769] Sending connection context
0925 09:19:40.590425 (+ 374us) client_negotiation.cc:240] Negotiation successful
0925 09:19:40.591154 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":362,"threads_started":1}
I20260925 09:19:40.594677 4165 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36275
I20260925 09:19:40.595005 4165 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:19:40.595909 4165 heartbeater.cc:511] Master 127.3.82.62:36275 requested a full tablet report, sending...
I20260925 09:19:40.598692 3976 ts_manager.cc:194] Registered new tserver with Master: ee737e16933d49f2aa9136ab8ded1e88 (127.3.82.1:45389)
I20260925 09:19:40.600919 3976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:41943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:19:40.622797 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36275
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:40.956780 4171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:40.957142 4171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:40.957326 4171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:40.965134 4171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:40.965327 4171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:40.965531 4171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:19:40.976788 4171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:40.978343 4171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:40.979822 4171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:40.994465 4176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:40.994444 4179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:40.994897 4177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:40.995621 4171 server_base.cc:1034] running on GCE node
I20260925 09:19:40.996533 4171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:40.997705 4171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:40.999029 4171 hybrid_clock.cc:648] HybridClock initialized: now 1790327980998935 us; error 88 us; skew 500 ppm
I20260925 09:19:40.999430 4171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:41 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:41.003556 4171 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:19:41.005492 4171 webserver.cc:460] Webserver started at http://127.3.82.2:41705/ using document root <none> and password file <none>
I20260925 09:19:41.006425 4171 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:41.006588 4171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:41.006975 4171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:41.011054 4171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "520697848b014702931475883bbe7b34"
format_stamp: "Formatted at 2026-09-25 09:19:41 on dist-test-slave-4wm5"
server_key: "2782827e2d66967088c58c9780339033"
server_key_iv: "b7ec4ec63a23d08c786f1608d73198d1"
server_key_version: "encryptionkey@0"
I20260925 09:19:41.012091 4171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "520697848b014702931475883bbe7b34"
format_stamp: "Formatted at 2026-09-25 09:19:41 on dist-test-slave-4wm5"
server_key: "2782827e2d66967088c58c9780339033"
server_key_iv: "b7ec4ec63a23d08c786f1608d73198d1"
server_key_version: "encryptionkey@0"
I20260925 09:19:41.019150 4171 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260925 09:19:41.023684 4186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:41.025568 4171 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:19:41.025810 4171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "520697848b014702931475883bbe7b34"
format_stamp: "Formatted at 2026-09-25 09:19:41 on dist-test-slave-4wm5"
server_key: "2782827e2d66967088c58c9780339033"
server_key_iv: "b7ec4ec63a23d08c786f1608d73198d1"
server_key_version: "encryptionkey@0"
I20260925 09:19:41.026137 4171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:41.038820 4171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:41.043934 4171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:41.044270 4171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:41.045398 4171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:41.047245 4171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:41.047367 4171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:41.047499 4171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:41.047573 4171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:41.085240 4171 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:36493
I20260925 09:19:41.085280 4298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:36493 every 8 connection(s)
I20260925 09:19:41.086706 4171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:19:41.092926 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4171
I20260925 09:19:41.093321 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:19:41.093973 3400 external_mini_cluster.cc:1428] Setting key 0da8a854074cbc5aa2efa6bdaa19ba19
Sep 25 09:19:41 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:41.110311 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.089590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:37467 (local address 127.3.82.62:36275)
0925 09:19:41.089865 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.089873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.091304 (+ 1431us) server_negotiation.cc:407] Connection header received
0925 09:19:41.093352 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.093361 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.093482 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.093617 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:41.095445 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.096209 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.097923 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.098379 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.098524 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.102952 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:41.102965 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:41.102969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:41.103006 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:41.106391 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:41.107749 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:41.107754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:41.107756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:41.107909 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:41.108422 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:41.108426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:41.108428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:41.108725 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:41.108892 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.109662 (+ 770us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.109952 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 09:19:41.111398 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.090210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36275 (local address 127.3.82.2:37467)
0925 09:19:41.091113 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.091162 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.092858 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.093964 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.093974 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.094636 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:41.095252 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.095271 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.096385 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.096392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.097722 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.097736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.098647 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.098654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.098902 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.099694 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:41.099719 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:41.102623 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:41.106602 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:41.106612 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:41.106629 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:41.107524 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:41.108044 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:41.108051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:41.108060 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:41.108278 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:41.108870 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:41.108878 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:41.109268 (+ 390us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.109555 (+ 287us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.110222 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":310,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:19:41.113273 4299 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36275
I20260925 09:19:41.113657 4299 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:41.114560 4299 heartbeater.cc:511] Master 127.3.82.62:36275 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:19:41.116385 3976 ts_manager.cc:194] Registered new tserver with Master: 520697848b014702931475883bbe7b34 (127.3.82.2:36493)
I20260925 09:19:41.117413 3976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:37467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:19:41.143015 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36275
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:41.486198 4304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:41.486508 4304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:41.486694 4304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:41.494410 4304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:41.494532 4304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:41.494753 4304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:19:41.505075 4304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:41.506575 4304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:41.508109 4304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:41.522642 4309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:41.522590 4310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:41.522661 4312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:41.523701 4304 server_base.cc:1034] running on GCE node
I20260925 09:19:41.524381 4304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:41.525425 4304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:41.526726 4304 hybrid_clock.cc:648] HybridClock initialized: now 1790327981526645 us; error 90 us; skew 500 ppm
I20260925 09:19:41.527221 4304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:41 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:41.531289 4304 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:19:41.533089 4304 webserver.cc:460] Webserver started at http://127.3.82.3:42865/ using document root <none> and password file <none>
I20260925 09:19:41.534013 4304 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:41.534188 4304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:41.534544 4304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:41.538146 4304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "add5aa4bfdc94aa998431eef82dd7412"
format_stamp: "Formatted at 2026-09-25 09:19:41 on dist-test-slave-4wm5"
server_key: "db7152bc55e127f7175042cec5273f26"
server_key_iv: "c5c1ec4dda6d0fb9cac370b8662c66ec"
server_key_version: "encryptionkey@0"
I20260925 09:19:41.539376 4304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "add5aa4bfdc94aa998431eef82dd7412"
format_stamp: "Formatted at 2026-09-25 09:19:41 on dist-test-slave-4wm5"
server_key: "db7152bc55e127f7175042cec5273f26"
server_key_iv: "c5c1ec4dda6d0fb9cac370b8662c66ec"
server_key_version: "encryptionkey@0"
I20260925 09:19:41.546412 4304 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260925 09:19:41.550729 4319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:41.552825 4304 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:19:41.552996 4304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "add5aa4bfdc94aa998431eef82dd7412"
format_stamp: "Formatted at 2026-09-25 09:19:41 on dist-test-slave-4wm5"
server_key: "db7152bc55e127f7175042cec5273f26"
server_key_iv: "c5c1ec4dda6d0fb9cac370b8662c66ec"
server_key_version: "encryptionkey@0"
I20260925 09:19:41.553200 4304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:41.570067 4304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:41.574867 4304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:41.575121 4304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:41.576090 4304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:41.577422 4304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:41.577493 4304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:41.577586 4304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:41.577639 4304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:41.605350 4165 heartbeater.cc:503] Master 127.3.82.62:36275 was elected leader, sending a full tablet report...
I20260925 09:19:41.614527 4304 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:36377
I20260925 09:19:41.614621 4431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:36377 every 8 connection(s)
I20260925 09:19:41.615837 4304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:19:41.625638 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4304
I20260925 09:19:41.625917 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:19:41.626355 3400 external_mini_cluster.cc:1428] Setting key f15b78967fcb0ddd3d7a68e4ef0d150c
Sep 25 09:19:41 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:41.638576 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.618552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:56851 (local address 127.3.82.62:36275)
0925 09:19:41.619090 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.619095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.619907 (+ 812us) server_negotiation.cc:407] Connection header received
0925 09:19:41.622090 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.622097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.622201 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.622332 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:41.624050 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.624913 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.626985 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.627447 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.627556 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.631965 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:41.631971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:41.631976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:41.632015 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:41.634956 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:41.635757 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:41.635761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:41.635763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:41.635894 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:41.636575 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:41.636578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:41.636581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:41.636891 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:41.637014 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.638023 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.638236 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":276,"threads_started":1}
I20260925 09:19:41.639524 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.618929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36275 (local address 127.3.82.3:56851)
0925 09:19:41.619702 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.619744 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.621714 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.622624 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.622635 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.623221 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:41.623849 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.623867 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.625098 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.625103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.626763 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.626777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.627703 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.627731 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.628051 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.628828 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:41.628855 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:41.631718 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:41.635118 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:41.635127 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:41.635147 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:41.635611 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:41.636185 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:41.636190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:41.636193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:41.636434 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:41.637073 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:41.637081 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:41.637451 (+ 370us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.637698 (+ 247us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.638304 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":307,"threads_started":1}
I20260925 09:19:41.641515 4432 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36275
I20260925 09:19:41.641928 4432 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:41.642863 4432 heartbeater.cc:511] Master 127.3.82.62:36275 requested a full tablet report, sending...
I20260925 09:19:41.644505 3976 ts_manager.cc:194] Registered new tserver with Master: add5aa4bfdc94aa998431eef82dd7412 (127.3.82.3:36377)
I20260925 09:19:41.645566 3976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:56851
I20260925 09:19:41.655782 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:19:41.676713 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.662952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.3.82.62:36275)
0925 09:19:41.663187 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.663195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.663830 (+ 635us) server_negotiation.cc:407] Connection header received
0925 09:19:41.663998 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.664003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.664095 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.664201 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:41.665868 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.666707 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.668078 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.668545 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.668664 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.671255 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:41.671260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:41.671264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:41.671305 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:41.674109 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:41.674797 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:41.674802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:41.674804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:41.674937 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:41.675265 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:41.675269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:41.675271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:41.675641 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:41.675834 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.676108 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.676297 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 09:19:41.683725 3976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:19:41.691517 3976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:41.728387 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.706245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.3.82.1:45389)
0925 09:19:41.719200 (+ 12955us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.719210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.719255 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:19:41.719368 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.719380 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.719708 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.719961 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:41.720809 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.722605 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.725467 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.727058 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.727225 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.727486 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.727828 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.728020 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12762,"thread_start_us":13918,"threads_started":1}
I20260925 09:19:41.728518 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.705986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:45389 (local address 127.0.0.1:52396)
0925 09:19:41.717137 (+ 11151us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.717208 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.717514 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.719999 (+ 2485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.720011 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.720069 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:41.720541 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.720558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.722780 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.722787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.725298 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.725313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.727263 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.727270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.727546 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.727617 (+ 71us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.727771 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.728062 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10712,"mutex_wait_us":91,"thread_start_us":11837,"threads_started":1}
I20260925 09:19:41.730147 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.706023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:36493 (local address 127.0.0.1:57348)
0925 09:19:41.718427 (+ 12404us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.718450 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.718578 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.720850 (+ 2272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.720858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.720900 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:41.721471 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.721490 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.724015 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.724020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.726608 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.726625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.729116 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.729126 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.729341 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.729377 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.729510 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.729663 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12257,"mutex_wait_us":30,"thread_start_us":12098,"threads_started":1}
I20260925 09:19:41.731113 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.706050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.3.82.2:36493)
0925 09:19:41.719896 (+ 13846us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.719905 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.719933 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:19:41.720052 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.720063 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.720315 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.720530 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:41.721776 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.723794 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.726820 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.728584 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.728725 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.728936 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.729576 (+ 640us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.729814 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13660,"thread_start_us":13566,"threads_started":1}
I20260925 09:19:41.732683 4100 tablet_service.cc:1467] Processing CreateTablet for tablet 77740b3b995c482b99cffef5b1d1b042 (DEFAULT_TABLE table=test-table [id=ed305e6dba214eed99bcad4a330131d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:41.733992 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.720529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:36377 (local address 127.0.0.1:41504)
0925 09:19:41.723777 (+ 3248us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.723797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.723934 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.724637 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.724644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.724670 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:41.725166 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.725181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.727423 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.727429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.730731 (+ 3302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.730744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.732878 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.732885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.733262 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.733299 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.733468 (+ 169us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.733607 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3116,"mutex_wait_us":38,"thread_start_us":2959,"threads_started":1}
I20260925 09:19:41.734416 4100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77740b3b995c482b99cffef5b1d1b042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:41.734803 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.719299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.3.82.3:36377)
0925 09:19:41.722372 (+ 3073us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.722381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.723919 (+ 1538us) server_negotiation.cc:407] Connection header received
0925 09:19:41.724087 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.724097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.724398 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.724625 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:41.725382 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.727190 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.730932 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.732632 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.732807 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.733059 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.734264 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.734508 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2887,"thread_start_us":3797,"threads_started":1}
I20260925 09:19:41.735324 4234 tablet_service.cc:1467] Processing CreateTablet for tablet 77740b3b995c482b99cffef5b1d1b042 (DEFAULT_TABLE table=test-table [id=ed305e6dba214eed99bcad4a330131d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:41.736887 4234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77740b3b995c482b99cffef5b1d1b042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:41.738373 4367 tablet_service.cc:1467] Processing CreateTablet for tablet 77740b3b995c482b99cffef5b1d1b042 (DEFAULT_TABLE table=test-table [id=ed305e6dba214eed99bcad4a330131d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:41.739571 4367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77740b3b995c482b99cffef5b1d1b042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:41.752252 4451 tablet_bootstrap.cc:492] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: Bootstrap starting.
I20260925 09:19:41.754827 4453 tablet_bootstrap.cc:492] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: Bootstrap starting.
I20260925 09:19:41.757745 4451 tablet_bootstrap.cc:654] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:41.758293 4453 tablet_bootstrap.cc:654] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:41.759274 4451 log.cc:824] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:41.760103 4452 tablet_bootstrap.cc:492] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: Bootstrap starting.
I20260925 09:19:41.760120 4453 log.cc:824] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:41.763399 4453 tablet_bootstrap.cc:492] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: No bootstrap required, opened a new log
I20260925 09:19:41.763717 4453 ts_tablet_manager.cc:1397] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 09:19:41.764590 4452 tablet_bootstrap.cc:654] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:41.764832 4451 tablet_bootstrap.cc:492] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: No bootstrap required, opened a new log
I20260925 09:19:41.765151 4451 ts_tablet_manager.cc:1397] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260925 09:19:41.766068 4452 log.cc:824] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:41.769601 4452 tablet_bootstrap.cc:492] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: No bootstrap required, opened a new log
I20260925 09:19:41.769936 4452 ts_tablet_manager.cc:1397] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260925 09:19:41.772221 4453 raft_consensus.cc:348] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.772652 4453 raft_consensus.cc:374] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:41.772749 4453 raft_consensus.cc:729] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520697848b014702931475883bbe7b34, State: Initialized, Role: FOLLOWER
I20260925 09:19:41.773427 4453 consensus_queue.cc:260] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.774648 4299 heartbeater.cc:503] Master 127.3.82.62:36275 was elected leader, sending a full tablet report...
I20260925 09:19:41.775456 4453 ts_tablet_manager.cc:1428] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260925 09:19:41.776082 4451 raft_consensus.cc:348] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.776556 4451 raft_consensus.cc:374] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:41.776662 4451 raft_consensus.cc:729] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee737e16933d49f2aa9136ab8ded1e88, State: Initialized, Role: FOLLOWER
I20260925 09:19:41.777328 4451 consensus_queue.cc:260] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.779253 4451 ts_tablet_manager.cc:1428] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260925 09:19:41.781092 4452 raft_consensus.cc:348] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.781565 4452 raft_consensus.cc:374] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:41.781697 4452 raft_consensus.cc:729] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add5aa4bfdc94aa998431eef82dd7412, State: Initialized, Role: FOLLOWER
I20260925 09:19:41.782374 4452 consensus_queue.cc:260] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.783532 4432 heartbeater.cc:503] Master 127.3.82.62:36275 was elected leader, sending a full tablet report...
I20260925 09:19:41.784286 4452 ts_tablet_manager.cc:1428] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260925 09:19:41.818640 4166 tablet.cc:2369] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:19:41.839246 4300 tablet.cc:2369] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:19:41.867482 4433 tablet.cc:2369] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:19:41.945401 4459 raft_consensus.cc:482] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:41.945791 4459 raft_consensus.cc:504] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.947872 4459 leader_election.cc:288] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 520697848b014702931475883bbe7b34 (127.3.82.2:36493), ee737e16933d49f2aa9136ab8ded1e88 (127.3.82.1:45389)
I20260925 09:19:41.954239 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.948454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:36493 (local address 127.3.82.3:55667)
0925 09:19:41.948670 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.948702 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.948841 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.949238 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.949243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.949272 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:41.949465 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.949474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.950634 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.950642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.952263 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.952280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.953421 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.953427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.953626 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.953652 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.953761 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.953936 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 09:19:41.954239 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.948442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:55667 (local address 127.3.82.2:36493)
0925 09:19:41.948930 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.948936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.948956 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:19:41.949039 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.949043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.949102 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.949209 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:41.949653 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.950501 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.952432 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.953243 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.953404 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.953515 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.953789 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.953963 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
I20260925 09:19:41.955407 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.948454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:45389 (local address 127.3.82.3:35191)
0925 09:19:41.949417 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:41.949433 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:41.949599 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:41.950006 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:41.950010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:41.950034 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:41.950446 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.950458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.951983 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.951988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.953337 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:41.953349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.954739 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.954744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:41.954930 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.954957 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 09:19:41.955036 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 09:19:41.955128 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":67,"thread_start_us":479,"threads_started":1}
I20260925 09:19:41.955585 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:41.948577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:35191 (local address 127.3.82.1:45389)
0925 09:19:41.948818 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:41.948826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:41.949539 (+ 713us) server_negotiation.cc:407] Connection header received
0925 09:19:41.949749 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:41.949761 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:41.949885 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:41.949993 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:41.950642 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.951823 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.953502 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:41.954499 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:41.954626 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:41.954733 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:41.955188 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 09:19:41.955334 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 09:19:41.955485 4254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77740b3b995c482b99cffef5b1d1b042" candidate_uuid: "add5aa4bfdc94aa998431eef82dd7412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520697848b014702931475883bbe7b34" is_pre_election: true
I20260925 09:19:41.956189 4254 raft_consensus.cc:2454] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add5aa4bfdc94aa998431eef82dd7412 in term 0.
I20260925 09:19:41.956666 4120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77740b3b995c482b99cffef5b1d1b042" candidate_uuid: "add5aa4bfdc94aa998431eef82dd7412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee737e16933d49f2aa9136ab8ded1e88" is_pre_election: true
I20260925 09:19:41.957062 4321 leader_election.cc:302] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520697848b014702931475883bbe7b34, add5aa4bfdc94aa998431eef82dd7412; no voters:
I20260925 09:19:41.957250 4120 raft_consensus.cc:2454] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add5aa4bfdc94aa998431eef82dd7412 in term 0.
I20260925 09:19:41.957597 4459 raft_consensus.cc:2781] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:41.957762 4459 raft_consensus.cc:482] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:41.957908 4459 raft_consensus.cc:3037] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:41.960439 4459 raft_consensus.cc:504] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.961112 4459 leader_election.cc:288] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [CANDIDATE]: Term 1 election: Requested vote from peers 520697848b014702931475883bbe7b34 (127.3.82.2:36493), ee737e16933d49f2aa9136ab8ded1e88 (127.3.82.1:45389)
I20260925 09:19:41.961760 4254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77740b3b995c482b99cffef5b1d1b042" candidate_uuid: "add5aa4bfdc94aa998431eef82dd7412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520697848b014702931475883bbe7b34"
I20260925 09:19:41.961890 4120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77740b3b995c482b99cffef5b1d1b042" candidate_uuid: "add5aa4bfdc94aa998431eef82dd7412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee737e16933d49f2aa9136ab8ded1e88"
I20260925 09:19:41.962081 4254 raft_consensus.cc:3037] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:41.962143 4120 raft_consensus.cc:3037] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:41.965739 4254 raft_consensus.cc:2454] T 77740b3b995c482b99cffef5b1d1b042 P 520697848b014702931475883bbe7b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add5aa4bfdc94aa998431eef82dd7412 in term 1.
I20260925 09:19:41.965739 4120 raft_consensus.cc:2454] T 77740b3b995c482b99cffef5b1d1b042 P ee737e16933d49f2aa9136ab8ded1e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add5aa4bfdc94aa998431eef82dd7412 in term 1.
I20260925 09:19:41.966352 4323 leader_election.cc:302] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: add5aa4bfdc94aa998431eef82dd7412, ee737e16933d49f2aa9136ab8ded1e88; no voters:
I20260925 09:19:41.966795 4459 raft_consensus.cc:2781] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:41.967324 4459 raft_consensus.cc:686] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [term 1 LEADER]: Becoming Leader. State: Replica: add5aa4bfdc94aa998431eef82dd7412, State: Running, Role: LEADER
I20260925 09:19:41.967871 4459 consensus_queue.cc:237] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } }
I20260925 09:19:41.973057 3975 catalog_manager.cc:5495] T 77740b3b995c482b99cffef5b1d1b042 P add5aa4bfdc94aa998431eef82dd7412 reported cstate change: term changed from 0 to 1, leader changed from <none> to add5aa4bfdc94aa998431eef82dd7412 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "add5aa4bfdc94aa998431eef82dd7412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520697848b014702931475883bbe7b34" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee737e16933d49f2aa9136ab8ded1e88" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 45389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "add5aa4bfdc94aa998431eef82dd7412" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36377 } health_report { overall_health: HEALTHY } } }
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:42.037377 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:42.018873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.3.82.1:45389)
0925 09:19:42.019180 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:42.019190 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:42.019726 (+ 536us) server_negotiation.cc:407] Connection header received
0925 09:19:42.019855 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:42.019860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:42.019969 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:42.020075 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:42.021791 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.022854 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.024251 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.024722 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.024840 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:42.029784 (+ 4944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:42.029804 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:42.029897 (+ 93us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:42.029972 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:42.033337 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.034610 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.034621 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.034624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.034810 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.035421 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.035436 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.035439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.035917 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:42.036329 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:42.036686 (+ 357us) server_negotiation.cc:299] Negotiation successful
0925 09:19:42.036956 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327982, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327982, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:42.094156 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:42.076221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.3.82.1:45389)
0925 09:19:42.076470 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:42.076480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:42.077087 (+ 607us) server_negotiation.cc:407] Connection header received
0925 09:19:42.077172 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:42.077181 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:42.077258 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:42.077367 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:42.079015 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.079841 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.081405 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.081987 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.082110 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:42.087043 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:42.087052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:42.087059 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:42.087117 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:42.090846 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.091719 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.091725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.091729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.091895 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.092466 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.092472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.092476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.092833 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:42.093114 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:42.093517 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 09:19:42.093775 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:42.146531 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:42.129358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.3.82.1:45389)
0925 09:19:42.129651 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:42.129657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:42.130220 (+ 563us) server_negotiation.cc:407] Connection header received
0925 09:19:42.130348 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:42.130353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:42.130429 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:42.130553 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:42.132518 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.133347 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.134907 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.135382 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.135490 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:42.140120 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:42.140128 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:42.140131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:42.140177 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:42.143326 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.144160 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.144165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.144168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.144320 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.144857 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.144862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.144864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.145263 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:42.145467 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:42.145753 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 09:19:42.146107 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260925 09:19:42.148434 4080 tablet_service.cc:2908] Rejecting scan request for tablet 77740b3b995c482b99cffef5b1d1b042: Uninitialized: clean time has not yet been initialized
I20260925 09:19:42.152209 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4037
I20260925 09:19:42.168270 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:42.153295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.3.82.1:45389)
0925 09:19:42.153509 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:42.153514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:42.153904 (+ 390us) server_negotiation.cc:407] Connection header received
0925 09:19:42.154157 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:42.154162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:42.154255 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:42.154350 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:42.155627 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.156599 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.158232 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.158693 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.158817 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:42.161663 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:42.161669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:42.161672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:42.161727 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:42.165114 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.166049 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.166055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.166057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.166216 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.166627 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.166632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.166634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.166985 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:42.167354 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:42.167795 (+ 441us) server_negotiation.cc:299] Negotiation successful
0925 09:19:42.167995 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:19:42.169060 4160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:42.266724 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4037
I20260925 09:19:42.284361 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4171
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:19:42.301366 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:42.285248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57358 (local address 127.3.82.2:36493)
0925 09:19:42.285430 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:42.285434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:42.285494 (+ 60us) server_negotiation.cc:407] Connection header received
0925 09:19:42.285563 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:42.285567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:42.285644 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:42.285749 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:42.287239 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.288059 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.289426 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.290150 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.290265 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:42.294598 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:42.294620 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:42.294637 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:42.294680 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:42.298276 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.299205 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.299214 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.299216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.299395 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.299816 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.299820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.299822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.300256 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:42.300510 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:42.300812 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 09:19:42.301078 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:19:42.302119 4294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:42.395747 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4171
I20260925 09:19:42.412842 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4304
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[3931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:19:42.430863 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:42.413602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.3.82.3:36377)
0925 09:19:42.414217 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:42.414221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:42.414238 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:19:42.414295 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:42.414299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:42.414371 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:42.414481 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:42.415790 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.416567 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.418271 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:42.418726 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:42.418838 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:42.423637 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:42.423654 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:42.423671 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:42.423730 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:42.427098 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.428170 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.428187 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.428194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.428389 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:42.428942 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:42.428949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:42.428952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:42.429409 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:42.429746 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:42.430287 (+ 541us) server_negotiation.cc:299] Negotiation successful
0925 09:19:42.430540 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":325,"threads_started":1}
I20260925 09:19:42.431677 4427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:42.527583 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4304
I20260925 09:19:42.544715 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 3944
I20260925 09:19:42.546188 4006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:42.657509 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 3944
2026-09-25T09:19:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4549 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[4482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[4482](info): set up 2 sockets
Sep 25 09:19:42 dist-test-slave-4wm5 krb5kdc[4482](info): commencing operation
krb5kdc: starting...
W20260925 09:19:43.778726 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.005s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:19:43 dist-test-slave-4wm5 krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:19:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:19:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:19:43.986115 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45195
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:44.325454 4495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:44.325891 4495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:44.326049 4495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:44.333528 4495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:19:44.333714 4495 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 09:19:44.333818 4495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:44.333971 4495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:44.334065 4495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:19:44.334159 4495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:19:44.344563 4495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45195
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:44.346122 4495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:44.347572 4495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:44.359673 4500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:44.359596 4503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:44.359635 4501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:44.360539 4495 server_base.cc:1034] running on GCE node
I20260925 09:19:44.361363 4495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:44.362967 4495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:44.364216 4495 hybrid_clock.cc:648] HybridClock initialized: now 1790327984364153 us; error 85 us; skew 500 ppm
I20260925 09:19:44.364610 4495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:44 dist-test-slave-4wm5 krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327984, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:44.367985 4495 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:19:44.369450 4495 webserver.cc:460] Webserver started at http://127.3.82.62:33239/ using document root <none> and password file <none>
I20260925 09:19:44.370241 4495 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:44.370333 4495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:44.370625 4495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:44.373787 4495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "e046fbed96f6441581cf3215389ec0c0"
format_stamp: "Formatted at 2026-09-25 09:19:44 on dist-test-slave-4wm5"
server_key: "4e8ef93c04c8a620c373768bce7c45ba"
server_key_iv: "eddac6bc65a4dc9321d382a4c681e60b"
server_key_version: "encryptionkey@0"
I20260925 09:19:44.374711 4495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "e046fbed96f6441581cf3215389ec0c0"
format_stamp: "Formatted at 2026-09-25 09:19:44 on dist-test-slave-4wm5"
server_key: "4e8ef93c04c8a620c373768bce7c45ba"
server_key_iv: "eddac6bc65a4dc9321d382a4c681e60b"
server_key_version: "encryptionkey@0"
I20260925 09:19:44.380385 4495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:19:44.384119 4510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:44.385761 4495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260925 09:19:44.386041 4495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "e046fbed96f6441581cf3215389ec0c0"
format_stamp: "Formatted at 2026-09-25 09:19:44 on dist-test-slave-4wm5"
server_key: "4e8ef93c04c8a620c373768bce7c45ba"
server_key_iv: "eddac6bc65a4dc9321d382a4c681e60b"
server_key_version: "encryptionkey@0"
I20260925 09:19:44.386307 4495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:44.407552 4495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:44.412686 4495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:44.413008 4495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:44.435920 4495 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45195
I20260925 09:19:44.435881 4571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45195 every 8 connection(s)
I20260925 09:19:44.437413 4495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:19:44.442077 4572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:44.446529 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4495
I20260925 09:19:44.446805 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:19:44.447606 3400 external_mini_cluster.cc:1428] Setting key 64a4d3162ee28c0ae9595ca1e4566f90
I20260925 09:19:44.456089 4572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: Bootstrap starting.
I20260925 09:19:44.460215 4572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:44.461578 4572 log.cc:824] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:44.464956 4572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: No bootstrap required, opened a new log
Sep 25 09:19:44 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:44.474591 4572 raft_consensus.cc:348] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e046fbed96f6441581cf3215389ec0c0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45195 } }
I20260925 09:19:44.475015 4572 raft_consensus.cc:374] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:44.475124 4572 raft_consensus.cc:729] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e046fbed96f6441581cf3215389ec0c0, State: Initialized, Role: FOLLOWER
I20260925 09:19:44.475780 4572 consensus_queue.cc:260] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e046fbed96f6441581cf3215389ec0c0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45195 } }
I20260925 09:19:44.476019 4572 raft_consensus.cc:388] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:19:44.476171 4572 raft_consensus.cc:482] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:19:44.476276 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:44.450215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.3.82.62:45195)
0925 09:19:44.451123 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:44.451146 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:44.451199 (+ 53us) server_negotiation.cc:407] Connection header received
0925 09:19:44.454191 (+ 2992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:44.454249 (+ 58us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:44.455026 (+ 777us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:44.455587 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:44.457137 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:44.459257 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:44.460695 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:44.461814 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:44.462021 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:44.467197 (+ 5176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:44.467212 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:44.467234 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:44.467279 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:44.471072 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:44.472104 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:44.472112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:44.472119 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:44.472348 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:44.472802 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:44.472806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:44.472808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:44.473448 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:44.473692 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:44.474239 (+ 547us) server_negotiation.cc:299] Negotiation successful
0925 09:19:44.474707 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":393,"threads_started":1}
I20260925 09:19:44.476372 4572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:44.478843 4572 raft_consensus.cc:504] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e046fbed96f6441581cf3215389ec0c0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45195 } }
I20260925 09:19:44.479465 4572 leader_election.cc:302] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e046fbed96f6441581cf3215389ec0c0; no voters:
I20260925 09:19:44.480155 4572 leader_election.cc:288] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:19:44.480316 4577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:44.481043 4577 raft_consensus.cc:686] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [term 1 LEADER]: Becoming Leader. State: Replica: e046fbed96f6441581cf3215389ec0c0, State: Running, Role: LEADER
I20260925 09:19:44.481637 4577 consensus_queue.cc:237] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e046fbed96f6441581cf3215389ec0c0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45195 } }
I20260925 09:19:44.482429 4572 sys_catalog.cc:564] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:19:44.484824 4579 sys_catalog.cc:455] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e046fbed96f6441581cf3215389ec0c0. Latest consensus state: current_term: 1 leader_uuid: "e046fbed96f6441581cf3215389ec0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e046fbed96f6441581cf3215389ec0c0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45195 } } }
I20260925 09:19:44.484817 4578 sys_catalog.cc:455] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e046fbed96f6441581cf3215389ec0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e046fbed96f6441581cf3215389ec0c0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45195 } } }
I20260925 09:19:44.485424 4578 sys_catalog.cc:458] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:44.485462 4579 sys_catalog.cc:458] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:44.487739 4584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:19:44.492960 4584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:19:44.501689 4584 catalog_manager.cc:1347] Generated new cluster ID: 7ea056e25a76456bb6576bef991e159a
I20260925 09:19:44.501860 4584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:19:44.515870 4584 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:19:44.517931 4584 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:19:44.529242 4584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: Generated new TSK 0
I20260925 09:19:44.530467 4584 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:19:44.595348 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45195
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:44.945084 4598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:44.945464 4598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:44.945634 4598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:44.953416 4598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:44.953549 4598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:44.953703 4598 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:19:44.953871 4598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:19:44.964293 4598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:44.965884 4598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:44.967317 4598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:44.982048 4604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:44.982131 4606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:44.982017 4603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:44.983072 4598 server_base.cc:1034] running on GCE node
I20260925 09:19:44.983947 4598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:44.985083 4598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:44.986325 4598 hybrid_clock.cc:648] HybridClock initialized: now 1790327984986239 us; error 66 us; skew 500 ppm
I20260925 09:19:44.986694 4598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:45 dist-test-slave-4wm5 krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:45.003667 4598 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:19:45.005477 4598 webserver.cc:460] Webserver started at http://127.3.82.1:40985/ using document root <none> and password file <none>
I20260925 09:19:45.006440 4598 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:45.006557 4598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:45.006870 4598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:45.010191 4598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "3eb1efb2d58845b9b2e2a08734f10660"
format_stamp: "Formatted at 2026-09-25 09:19:45 on dist-test-slave-4wm5"
server_key: "dfb6b7fbb7c59804229c0450556dcf27"
server_key_iv: "a6de1642495f63aefeb1ac010a3bf697"
server_key_version: "encryptionkey@0"
I20260925 09:19:45.011031 4598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "3eb1efb2d58845b9b2e2a08734f10660"
format_stamp: "Formatted at 2026-09-25 09:19:45 on dist-test-slave-4wm5"
server_key: "dfb6b7fbb7c59804229c0450556dcf27"
server_key_iv: "a6de1642495f63aefeb1ac010a3bf697"
server_key_version: "encryptionkey@0"
I20260925 09:19:45.016847 4598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:19:45.020826 4613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:45.022560 4598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:19:45.022748 4598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "3eb1efb2d58845b9b2e2a08734f10660"
format_stamp: "Formatted at 2026-09-25 09:19:45 on dist-test-slave-4wm5"
server_key: "dfb6b7fbb7c59804229c0450556dcf27"
server_key_iv: "a6de1642495f63aefeb1ac010a3bf697"
server_key_version: "encryptionkey@0"
I20260925 09:19:45.022989 4598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:45.054497 4598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:45.059248 4598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:45.059557 4598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:45.062384 4598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:45.062469 4598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:45.062605 4598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:45.062693 4598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:19:45 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:45.089337 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.063357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:36479 (local address 127.3.82.62:45195)
0925 09:19:45.063859 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:45.063868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:45.065250 (+ 1382us) server_negotiation.cc:407] Connection header received
0925 09:19:45.067817 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:45.067823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:45.067926 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:45.068065 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:45.071023 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.072086 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.074602 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.075198 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.075350 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.080797 (+ 5447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:45.080807 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:45.080814 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:45.080868 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:45.084691 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.085897 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.085904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.085907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.086085 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.086718 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.086725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.086731 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.087158 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:45.087312 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:45.088521 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0925 09:19:45.088884 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260925 09:19:45.090600 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.064089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.3.82.1:36479)
0925 09:19:45.065040 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:45.065095 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:45.067421 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:45.068417 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:45.068436 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:45.069502 (+ 1066us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:45.070739 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.070781 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.072268 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.072277 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.074393 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.074420 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.075476 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.075488 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.075770 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.076718 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:45.076760 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:45.080489 (+ 3729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:45.084861 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.084872 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.084896 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.085676 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.086236 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.086241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.086245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.086565 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.087400 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:45.087408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:45.087918 (+ 510us) client_negotiation.cc:769] Sending connection context
0925 09:19:45.088275 (+ 357us) client_negotiation.cc:240] Negotiation successful
0925 09:19:45.088921 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":447,"threads_started":1}
I20260925 09:19:45.116189 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.098204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.3.82.1:45567)
0925 09:19:45.102511 (+ 4307us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:45.102536 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:45.102763 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:45.103154 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:45.103159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:45.103673 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:45.103980 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.103990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.105074 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.105080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.106510 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.106531 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.107610 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.107617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.107822 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.108633 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:45.108651 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:45.109530 (+ 879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:45.112972 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.112978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.112982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.113650 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.114229 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.114234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.114238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.114410 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.115241 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:45.115252 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:45.115444 (+ 192us) client_negotiation.cc:769] Sending connection context
0925 09:19:45.115572 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 09:19:45.115838 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4132,"thread_start_us":3905,"threads_started":1}
I20260925 09:19:45.116597 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.098757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:45567 (local address 127.3.82.62:45195)
0925 09:19:45.098974 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:45.098984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:45.102668 (+ 3684us) server_negotiation.cc:407] Connection header received
0925 09:19:45.102931 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:45.102936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:45.103006 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:45.103122 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:45.104163 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.104913 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.106742 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.107394 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.107570 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.109752 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:45.109757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:45.109760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:45.109793 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:45.112806 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.113894 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.113900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.113906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.114086 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.114580 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.114588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.114595 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.115082 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:45.115270 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:45.115794 (+ 524us) server_negotiation.cc:299] Negotiation successful
0925 09:19:45.116124 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:19:45.121563 4598 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:46143
I20260925 09:19:45.121584 4733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:46143 every 8 connection(s)
I20260925 09:19:45.122874 4598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:19:45.125008 4734 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45195
I20260925 09:19:45.125255 4734 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:45.126055 4734 heartbeater.cc:511] Master 127.3.82.62:45195 requested a full tablet report, sending...
I20260925 09:19:45.128492 4527 ts_manager.cc:194] Registered new tserver with Master: 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1:46143)
I20260925 09:19:45.128732 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4598
I20260925 09:19:45.128942 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:19:45.129577 3400 external_mini_cluster.cc:1428] Setting key f59c9dd19defb22e08b62e7a7f47e50d
I20260925 09:19:45.130254 4527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:36479
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:19:45.178922 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45195
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:45.516419 4738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:45.516722 4738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:45.516821 4738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:45.524113 4738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:45.524243 4738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:45.524326 4738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:19:45.524408 4738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:19:45.534807 4738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:45.536103 4738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:45.537566 4738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:45.551647 4746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:45.551707 4743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:45.551755 4744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:45.552701 4738 server_base.cc:1034] running on GCE node
I20260925 09:19:45.553365 4738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:45.554431 4738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:45.555689 4738 hybrid_clock.cc:648] HybridClock initialized: now 1790327985555596 us; error 61 us; skew 500 ppm
I20260925 09:19:45.556172 4738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:45 dist-test-slave-4wm5 krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:45.559784 4738 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:19:45.561543 4738 webserver.cc:460] Webserver started at http://127.3.82.2:37539/ using document root <none> and password file <none>
I20260925 09:19:45.562373 4738 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:45.562513 4738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:45.562857 4738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:45.566076 4738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "a97b1a609d2642a5a553647bc3723627"
format_stamp: "Formatted at 2026-09-25 09:19:45 on dist-test-slave-4wm5"
server_key: "ad6bb03983e7eff348007ee154771015"
server_key_iv: "089a7b77a1a9737ed8f2b3e6c81dafa9"
server_key_version: "encryptionkey@0"
I20260925 09:19:45.566943 4738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "a97b1a609d2642a5a553647bc3723627"
format_stamp: "Formatted at 2026-09-25 09:19:45 on dist-test-slave-4wm5"
server_key: "ad6bb03983e7eff348007ee154771015"
server_key_iv: "089a7b77a1a9737ed8f2b3e6c81dafa9"
server_key_version: "encryptionkey@0"
I20260925 09:19:45.572726 4738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:19:45.576786 4753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:45.578531 4738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:19:45.578751 4738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "a97b1a609d2642a5a553647bc3723627"
format_stamp: "Formatted at 2026-09-25 09:19:45 on dist-test-slave-4wm5"
server_key: "ad6bb03983e7eff348007ee154771015"
server_key_iv: "089a7b77a1a9737ed8f2b3e6c81dafa9"
server_key_version: "encryptionkey@0"
I20260925 09:19:45.579002 4738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:45.601641 4738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:45.606563 4738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:45.606853 4738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:45.609905 4738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:45.610041 4738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:45.610221 4738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:45.610301 4738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:19:45 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:45.640259 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.611505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.3.82.2:50837)
0925 09:19:45.612449 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:45.612507 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:45.615610 (+ 3103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:45.617167 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:45.617188 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:45.618571 (+ 1383us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:45.619791 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.619832 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.621250 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.621255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.624242 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.624266 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.625191 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.625202 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.625608 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.626762 (+ 1154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:45.626799 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:45.630730 (+ 3931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:45.634033 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.634043 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.634068 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.634807 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.635367 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.635373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.635380 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.635598 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.636403 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:45.636409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:45.636983 (+ 574us) client_negotiation.cc:769] Sending connection context
0925 09:19:45.637307 (+ 324us) client_negotiation.cc:240] Negotiation successful
0925 09:19:45.638019 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":363,"threads_started":1}
I20260925 09:19:45.641630 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.610855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:50837 (local address 127.3.82.62:45195)
0925 09:19:45.611078 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:45.611088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:45.612685 (+ 1597us) server_negotiation.cc:407] Connection header received
0925 09:19:45.616108 (+ 3423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:45.616119 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:45.616230 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:45.616328 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:45.620113 (+ 3785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.621083 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.624477 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.624894 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.625002 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.631059 (+ 6057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:45.631065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:45.631068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:45.631107 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:45.633803 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.635054 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.635064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.635070 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.635242 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.635747 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.635752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.635756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.636171 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:45.636325 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:45.641131 (+ 4806us) server_negotiation.cc:299] Negotiation successful
0925 09:19:45.641354 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:19:45.664804 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.649051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.3.82.2:53039)
0925 09:19:45.650091 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:45.650115 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:45.650320 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:45.650747 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:45.650752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:45.651305 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:45.651621 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.651632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.652761 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.652768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.654101 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:45.654124 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.655489 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.655500 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:45.655665 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.656618 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:45.656635 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:45.657945 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:45.661645 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.661650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.661662 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.662358 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.662757 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:45.662763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:45.662767 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:45.662954 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:45.663885 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:45.663896 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:45.664075 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 09:19:45.664193 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 09:19:45.664452 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":552,"threads_started":1}
I20260925 09:19:45.665290 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:45.649073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:53039 (local address 127.3.82.62:45195)
0925 09:19:45.649316 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:45.649325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:45.650237 (+ 912us) server_negotiation.cc:407] Connection header received
0925 09:19:45.650517 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:45.650523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:45.650596 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:45.650720 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:45.651789 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.652595 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.654328 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:45.654984 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:45.655079 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:45.658168 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:45.658175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:45.658179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:45.658216 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:45.661477 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.662497 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.662504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.662507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.662639 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:45.663130 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:45.663139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:45.663145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:45.663730 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:45.663918 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:45.664383 (+ 465us) server_negotiation.cc:299] Negotiation successful
0925 09:19:45.664810 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:19:45.672693 4738 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:36801
I20260925 09:19:45.672730 4872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:36801 every 8 connection(s)
I20260925 09:19:45.674026 4738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:19:45.676354 4873 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45195
I20260925 09:19:45.676602 4873 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:45.677248 4873 heartbeater.cc:511] Master 127.3.82.62:45195 requested a full tablet report, sending...
I20260925 09:19:45.678673 4526 ts_manager.cc:194] Registered new tserver with Master: a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801)
I20260925 09:19:45.679682 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4738
I20260925 09:19:45.679669 4526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:50837
I20260925 09:19:45.679878 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:19:45.680384 3400 external_mini_cluster.cc:1428] Setting key 87419a13a9cdc5d9622a54cb7e5d3a3f
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:19:45.726503 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45195
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:46.064085 4877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:46.064391 4877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:46.064491 4877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:46.072021 4877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:46.072152 4877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:46.072237 4877 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:19:46.072326 4877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:19:46.082731 4877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:45195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:46.084218 4877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:46.085557 4877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:46.100760 4885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:46.100842 4883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:46.100795 4882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:46.102264 4877 server_base.cc:1034] running on GCE node
I20260925 09:19:46.102891 4877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:46.103856 4877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:46.105113 4877 hybrid_clock.cc:648] HybridClock initialized: now 1790327986105035 us; error 43 us; skew 500 ppm
I20260925 09:19:46.105440 4877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:46 dist-test-slave-4wm5 krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:46.108914 4877 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:19:46.110594 4877 webserver.cc:460] Webserver started at http://127.3.82.3:37989/ using document root <none> and password file <none>
I20260925 09:19:46.111334 4877 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:46.111423 4877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:46.111699 4877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:46.114766 4877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "ff1074a5f7d64ca584665210f28e2d12"
format_stamp: "Formatted at 2026-09-25 09:19:46 on dist-test-slave-4wm5"
server_key: "ec16836bfe474a5b767e0a6a227eddcb"
server_key_iv: "595905b2592504a806e24450af50e57e"
server_key_version: "encryptionkey@0"
I20260925 09:19:46.115543 4877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "ff1074a5f7d64ca584665210f28e2d12"
format_stamp: "Formatted at 2026-09-25 09:19:46 on dist-test-slave-4wm5"
server_key: "ec16836bfe474a5b767e0a6a227eddcb"
server_key_iv: "595905b2592504a806e24450af50e57e"
server_key_version: "encryptionkey@0"
I20260925 09:19:46.121145 4877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:19:46.125345 4892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:46.127024 4877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:19:46.127242 4877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "ff1074a5f7d64ca584665210f28e2d12"
format_stamp: "Formatted at 2026-09-25 09:19:46 on dist-test-slave-4wm5"
server_key: "ec16836bfe474a5b767e0a6a227eddcb"
server_key_iv: "595905b2592504a806e24450af50e57e"
server_key_version: "encryptionkey@0"
I20260925 09:19:46.127525 4877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:46.133807 4734 heartbeater.cc:503] Master 127.3.82.62:45195 was elected leader, sending a full tablet report...
I20260925 09:19:46.147920 4877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:46.152894 4877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:46.153223 4877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:46.156391 4877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:46.156567 4877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:46.156741 4877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:46.156826 4877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:19:46 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:46.183707 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.157349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50637 (local address 127.3.82.62:45195)
0925 09:19:46.157594 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.157601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.159307 (+ 1706us) server_negotiation.cc:407] Connection header received
0925 09:19:46.162771 (+ 3464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.162779 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.162898 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.163028 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:46.166217 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.167076 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.169426 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.170053 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.170238 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.175141 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:46.175153 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:46.175161 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:46.175222 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:46.179084 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.180034 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.180042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.180050 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.180200 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.180920 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.180931 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.180933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.181376 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:46.181559 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.182999 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.183373 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260925 09:19:46.185920 4901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.158090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.3.82.3:50637)
0925 09:19:46.159079 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.159140 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.162269 (+ 3129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.163416 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.163432 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.164546 (+ 1114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:46.165909 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.165949 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.167305 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.167310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.169192 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.169213 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.170384 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.170395 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.170643 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.171494 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:46.171522 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:46.174722 (+ 3200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:46.179288 (+ 4566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:46.179300 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:46.179329 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:46.179858 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:46.180327 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:46.180336 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:46.180340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:46.180786 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:46.181546 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:46.181561 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:46.182356 (+ 795us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.182849 (+ 493us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.183674 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":372,"threads_started":1}
I20260925 09:19:46.209564 4877 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:36265
I20260925 09:19:46.209585 5011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:36265 every 8 connection(s)
I20260925 09:19:46.211063 4877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:19:46.213713 5012 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45195
I20260925 09:19:46.214033 5012 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:46.214967 5012 heartbeater.cc:511] Master 127.3.82.62:45195 requested a full tablet report, sending...
I20260925 09:19:46.216741 4526 ts_manager.cc:194] Registered new tserver with Master: ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265)
I20260925 09:19:46.217182 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.199516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:34581 (local address 127.3.82.62:45195)
0925 09:19:46.199755 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.199765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.200396 (+ 631us) server_negotiation.cc:407] Connection header received
0925 09:19:46.200722 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.200729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.200828 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.200890 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:46.202965 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.204052 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.206290 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.207040 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.207213 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.210029 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:46.210037 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:46.210042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:46.210088 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:46.213771 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.214700 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.214706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.214710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.214884 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.215457 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.215463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.215466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.215882 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:46.216017 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.216609 (+ 592us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.216828 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:19:46.217295 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.199467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.3.82.3:34581)
0925 09:19:46.200252 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.200280 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.200543 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.201161 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.201167 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.202039 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:46.202774 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.202788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.204212 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.204219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.206089 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.206107 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.207304 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.207317 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.207491 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.208355 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:46.208382 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:46.209750 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:46.213952 (+ 4202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:46.213958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:46.213960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:46.214542 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:46.215140 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:46.215147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:46.215157 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:46.215314 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:46.216036 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:46.216044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:46.216284 (+ 240us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.216462 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.216869 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":467,"threads_started":1}
I20260925 09:19:46.217988 4526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:50637
I20260925 09:19:46.217991 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 4877
I20260925 09:19:46.218178 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:19:46.218709 3400 external_mini_cluster.cc:1428] Setting key c63ca941d46d60715c5420400854f7e1
I20260925 09:19:46.222875 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:19:46 dist-test-slave-4wm5 krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:19:46 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:46.276154 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.259864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.3.82.62:45195)
0925 09:19:46.260064 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.260069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.260507 (+ 438us) server_negotiation.cc:407] Connection header received
0925 09:19:46.260656 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.260661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.260734 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.260838 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:46.262555 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.263347 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.264805 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.265226 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.265324 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.269782 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:46.269790 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:46.269793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:46.269909 (+ 116us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:46.273031 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.273937 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.273942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.273945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.274188 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.274674 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.274678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.274680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.274984 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:46.275197 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.275599 (+ 402us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.275789 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:19:46.281008 4527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:19:46.288508 4527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:46.326498 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.302959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53648 (local address 127.3.82.1:46143)
0925 09:19:46.315686 (+ 12727us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.315697 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.316549 (+ 852us) server_negotiation.cc:407] Connection header received
0925 09:19:46.316689 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.316702 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.317081 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.318470 (+ 1389us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.318772 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.320799 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.324056 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.325097 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.325257 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.325512 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.325975 (+ 463us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.326173 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12490,"thread_start_us":12749,"threads_started":1}
I20260925 09:19:46.326532 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.302721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46143 (local address 127.0.0.1:53648)
0925 09:19:46.315889 (+ 13168us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.315941 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.316269 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.317445 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.317458 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.317506 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.318162 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.318183 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.320989 (+ 2806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.320999 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.323845 (+ 2846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.323862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.325316 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.325322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.325610 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.325677 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.325877 (+ 200us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.326099 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12851,"thread_start_us":19877,"threads_started":1}
I20260925 09:19:46.330458 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.302849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:36265 (local address 127.0.0.1:49778)
0925 09:19:46.321041 (+ 18192us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.321085 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.321302 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.322259 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.322267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.322308 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.322958 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.322976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.325324 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.325328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.327558 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.327572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.329387 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.329394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.329630 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.329707 (+ 77us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.329814 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.330004 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18003,"mutex_wait_us":1,"thread_start_us":12606,"threads_started":1}
I20260925 09:19:46.330456 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.303579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.3.82.3:36265)
0925 09:19:46.320631 (+ 17052us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.320644 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.321491 (+ 847us) server_negotiation.cc:407] Connection header received
0925 09:19:46.321640 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.321650 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.321979 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.323151 (+ 1172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.323488 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.325184 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.327825 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.329140 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.329319 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.329589 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.329907 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.330105 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16827,"thread_start_us":16642,"threads_started":1}
I20260925 09:19:46.332454 4664 tablet_service.cc:1467] Processing CreateTablet for tablet 4eec0d3b325e4149beb0cc2ce689da99 (DEFAULT_TABLE table=test-table [id=3b4d0642ff0f4ae1bae60593d4b2e629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:46.333904 5025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.303245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:36801 (local address 127.0.0.1:47650)
0925 09:19:46.322931 (+ 19686us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.322957 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.323663 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.324645 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.324650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.324676 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.325043 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.325053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.327379 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.327389 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.329487 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.329504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.332919 (+ 3415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.332926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.333185 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.333222 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.333329 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.333498 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19523,"mutex_wait_us":63,"thread_start_us":17422,"threads_started":1}
I20260925 09:19:46.334188 4664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eec0d3b325e4149beb0cc2ce689da99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.334225 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.303834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.3.82.2:36801)
0925 09:19:46.322205 (+ 18371us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.322216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.323073 (+ 857us) server_negotiation.cc:407] Connection header received
0925 09:19:46.323895 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.323904 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.324212 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.324391 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.325251 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.327229 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.329740 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.332615 (+ 2875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.332797 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.333030 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.333461 (+ 431us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.333661 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17954,"thread_start_us":18013,"threads_started":1}
I20260925 09:19:46.335184 4942 tablet_service.cc:1467] Processing CreateTablet for tablet 4eec0d3b325e4149beb0cc2ce689da99 (DEFAULT_TABLE table=test-table [id=3b4d0642ff0f4ae1bae60593d4b2e629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:46.337136 4942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eec0d3b325e4149beb0cc2ce689da99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.338006 4803 tablet_service.cc:1467] Processing CreateTablet for tablet 4eec0d3b325e4149beb0cc2ce689da99 (DEFAULT_TABLE table=test-table [id=3b4d0642ff0f4ae1bae60593d4b2e629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:46.339197 4803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eec0d3b325e4149beb0cc2ce689da99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.350229 5031 tablet_bootstrap.cc:492] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Bootstrap starting.
I20260925 09:19:46.352371 5032 tablet_bootstrap.cc:492] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Bootstrap starting.
I20260925 09:19:46.354094 5033 tablet_bootstrap.cc:492] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Bootstrap starting.
I20260925 09:19:46.354868 5031 tablet_bootstrap.cc:654] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.356299 5032 tablet_bootstrap.cc:654] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.357594 5031 log.cc:824] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:46.357616 5032 log.cc:824] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:46.358053 5033 tablet_bootstrap.cc:654] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.359650 5033 log.cc:824] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:46.363557 5033 tablet_bootstrap.cc:492] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: No bootstrap required, opened a new log
I20260925 09:19:46.363924 5033 ts_tablet_manager.cc:1397] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260925 09:19:46.363965 5032 tablet_bootstrap.cc:492] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: No bootstrap required, opened a new log
I20260925 09:19:46.363977 5031 tablet_bootstrap.cc:492] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: No bootstrap required, opened a new log
I20260925 09:19:46.364238 5032 ts_tablet_manager.cc:1397] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260925 09:19:46.364238 5031 ts_tablet_manager.cc:1397] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260925 09:19:46.372007 5033 raft_consensus.cc:348] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.372519 5033 raft_consensus.cc:374] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.372637 5033 raft_consensus.cc:729] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1074a5f7d64ca584665210f28e2d12, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.373338 5033 consensus_queue.cc:260] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.374635 5012 heartbeater.cc:503] Master 127.3.82.62:45195 was elected leader, sending a full tablet report...
I20260925 09:19:46.375389 5033 ts_tablet_manager.cc:1428] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260925 09:19:46.375218 5031 raft_consensus.cc:348] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.375697 5031 raft_consensus.cc:374] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.375798 5031 raft_consensus.cc:729] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb1efb2d58845b9b2e2a08734f10660, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.376579 5031 consensus_queue.cc:260] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.376648 5032 raft_consensus.cc:348] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.377296 5032 raft_consensus.cc:374] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.377473 5032 raft_consensus.cc:729] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97b1a609d2642a5a553647bc3723627, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.378500 5031 ts_tablet_manager.cc:1428] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260925 09:19:46.378306 5032 consensus_queue.cc:260] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.379601 4873 heartbeater.cc:503] Master 127.3.82.62:45195 was elected leader, sending a full tablet report...
I20260925 09:19:46.380429 5032 ts_tablet_manager.cc:1428] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260925 09:19:46.408775 5038 raft_consensus.cc:482] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:46.409088 5038 raft_consensus.cc:504] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.411082 5038 leader_election.cc:288] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265), a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801)
I20260925 09:19:46.418579 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.411627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:37899 (local address 127.3.82.2:36801)
0925 09:19:46.411808 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.411815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.412925 (+ 1110us) server_negotiation.cc:407] Connection header received
0925 09:19:46.413194 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.413202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.413274 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.413381 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.414000 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.414730 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.416653 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.417464 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.417651 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.417780 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.418131 (+ 351us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.418273 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:19:46.418841 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.411636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:36801 (local address 127.3.82.1:37899)
0925 09:19:46.412764 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.412785 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.413021 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.413398 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.413402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.413443 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.413788 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.413799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.414877 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.414882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.416475 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.416489 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.417714 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.417721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.417954 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.417980 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.418090 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.418249 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"spinlock_wait_cycles":30464,"thread_start_us":1356,"threads_started":1}
I20260925 09:19:46.419378 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.411552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:33599 (local address 127.3.82.3:36265)
0925 09:19:46.411780 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.411787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.413484 (+ 1697us) server_negotiation.cc:407] Connection header received
0925 09:19:46.413723 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.413742 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.413808 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.413970 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.414598 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.415340 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.417233 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.418177 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.418354 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.418502 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.418939 (+ 437us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.419103 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:19:46.419399 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.411636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:36265 (local address 127.3.82.1:33599)
0925 09:19:46.413342 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.413360 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.413557 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.413988 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.413993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.414019 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.414394 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.414404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.415541 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.415546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.417022 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.417042 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.418459 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.418469 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.418647 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.418712 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.418873 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.419035 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1557,"mutex_wait_us":33,"thread_start_us":799,"threads_started":1}
I20260925 09:19:46.420073 4823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eec0d3b325e4149beb0cc2ce689da99" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a97b1a609d2642a5a553647bc3723627" is_pre_election: true
I20260925 09:19:46.420535 4962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eec0d3b325e4149beb0cc2ce689da99" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1074a5f7d64ca584665210f28e2d12" is_pre_election: true
I20260925 09:19:46.420710 4823 raft_consensus.cc:2454] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 0.
I20260925 09:19:46.421088 4962 raft_consensus.cc:2454] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 0.
I20260925 09:19:46.421491 4617 leader_election.cc:302] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb1efb2d58845b9b2e2a08734f10660, a97b1a609d2642a5a553647bc3723627; no voters:
I20260925 09:19:46.421967 5038 raft_consensus.cc:2781] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:46.422122 5038 raft_consensus.cc:482] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:46.422235 5038 raft_consensus.cc:3037] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:46.424520 5038 raft_consensus.cc:504] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.425206 5038 leader_election.cc:288] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 election: Requested vote from peers ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265), a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801)
I20260925 09:19:46.425856 4823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eec0d3b325e4149beb0cc2ce689da99" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a97b1a609d2642a5a553647bc3723627"
I20260925 09:19:46.425912 4962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eec0d3b325e4149beb0cc2ce689da99" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1074a5f7d64ca584665210f28e2d12"
I20260925 09:19:46.426110 4962 raft_consensus.cc:3037] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:46.426110 4823 raft_consensus.cc:3037] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:19:46.426345 4874 tablet.cc:2369] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:19:46.429570 4962 raft_consensus.cc:2454] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 1.
I20260925 09:19:46.429570 4823 raft_consensus.cc:2454] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 1.
I20260925 09:19:46.430219 4617 leader_election.cc:302] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3eb1efb2d58845b9b2e2a08734f10660, a97b1a609d2642a5a553647bc3723627, ff1074a5f7d64ca584665210f28e2d12; no voters:
I20260925 09:19:46.430629 5038 raft_consensus.cc:2781] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:46.431090 5038 raft_consensus.cc:686] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 LEADER]: Becoming Leader. State: Replica: 3eb1efb2d58845b9b2e2a08734f10660, State: Running, Role: LEADER
I20260925 09:19:46.431651 5038 consensus_queue.cc:237] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } }
I20260925 09:19:46.437690 4526 catalog_manager.cc:5495] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } health_report { overall_health: UNKNOWN } } }
W20260925 09:19:46.463312 5013 tablet.cc:2369] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:19:46.472275 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.466217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.3.82.1:46143)
0925 09:19:46.466468 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.466477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.466580 (+ 103us) server_negotiation.cc:407] Connection header received
0925 09:19:46.466930 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.466935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.467031 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.467189 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:46.467597 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.468301 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.469706 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.470244 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.470386 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.471004 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:46.471578 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:46.471746 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.471893 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.471986 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 09:19:46.481534 4962 raft_consensus.cc:1261] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Refusing update from remote peer 3eb1efb2d58845b9b2e2a08734f10660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:46.481528 4823 raft_consensus.cc:1261] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Refusing update from remote peer 3eb1efb2d58845b9b2e2a08734f10660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:46.482743 5042 consensus_queue.cc:1035] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:46.483177 5038 consensus_queue.cc:1035] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:46.522090 4526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:46.522408 4526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:46.526350 4526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: Sending DeleteTablet for 3 replicas of tablet 4eec0d3b325e4149beb0cc2ce689da99
I20260925 09:19:46.527899 4664 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eec0d3b325e4149beb0cc2ce689da99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:46 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53648
I20260925 09:19:46.527917 4942 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eec0d3b325e4149beb0cc2ce689da99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:46 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:49778
I20260925 09:19:46.528687 4803 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eec0d3b325e4149beb0cc2ce689da99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:46 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:47650
I20260925 09:19:46.530408 4526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:19:46.531466 4526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:46.534231 5056 tablet_replica.cc:330] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: stopping tablet replica
I20260925 09:19:46.534791 5056 raft_consensus.cc:2229] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:19:46.535602 5056 raft_consensus.cc:2258] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:46.542207 5056 ts_tablet_manager.cc:1905] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:46.542866 4941 tablet_service.cc:1467] Processing CreateTablet for tablet dfd650989e904e4ab0d44aa573127ad8 (DEFAULT_TABLE table=test-table [id=7f2116ab74b74fc4aaa77f807ee08fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:46.543574 4941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd650989e904e4ab0d44aa573127ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.541633 4802 tablet_service.cc:1467] Processing CreateTablet for tablet dfd650989e904e4ab0d44aa573127ad8 (DEFAULT_TABLE table=test-table [id=7f2116ab74b74fc4aaa77f807ee08fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:46.550501 4802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd650989e904e4ab0d44aa573127ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.553316 5057 tablet_replica.cc:330] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: stopping tablet replica
I20260925 09:19:46.554323 5057 raft_consensus.cc:2229] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:46.554630 5055 tablet_replica.cc:330] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: stopping tablet replica
I20260925 09:19:46.555133 5055 raft_consensus.cc:2229] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:46.555902 5057 raft_consensus.cc:2258] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:46.556075 5056 ts_tablet_manager.cc:1918] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:19:46.556355 5056 log.cc:1196] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/4eec0d3b325e4149beb0cc2ce689da99
I20260925 09:19:46.556908 5055 raft_consensus.cc:2258] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:46.556994 5056 ts_tablet_manager.cc:1939] T 4eec0d3b325e4149beb0cc2ce689da99 P 3eb1efb2d58845b9b2e2a08734f10660: Deleting consensus metadata
I20260925 09:19:46.558307 4664 tablet_service.cc:1467] Processing CreateTablet for tablet dfd650989e904e4ab0d44aa573127ad8 (DEFAULT_TABLE table=test-table [id=7f2116ab74b74fc4aaa77f807ee08fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:46.558859 4664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfd650989e904e4ab0d44aa573127ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.559324 5057 ts_tablet_manager.cc:1905] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:46.559861 4512 catalog_manager.cc:4841] TS 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1:46143): tablet 4eec0d3b325e4149beb0cc2ce689da99 (table test-table [id=3b4d0642ff0f4ae1bae60593d4b2e629]) successfully deleted
I20260925 09:19:46.561213 5055 ts_tablet_manager.cc:1905] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:46.562932 5033 tablet_bootstrap.cc:492] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Bootstrap starting.
I20260925 09:19:46.563243 5032 tablet_bootstrap.cc:492] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Bootstrap starting.
I20260925 09:19:46.564860 5033 tablet_bootstrap.cc:654] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.565083 5032 tablet_bootstrap.cc:654] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.569355 5031 tablet_bootstrap.cc:492] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Bootstrap starting.
I20260925 09:19:46.571973 5031 tablet_bootstrap.cc:654] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.585322 5033 tablet_bootstrap.cc:492] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: No bootstrap required, opened a new log
I20260925 09:19:46.585605 5033 ts_tablet_manager.cc:1397] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.003s
I20260925 09:19:46.586580 5033 raft_consensus.cc:348] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.586951 5033 raft_consensus.cc:374] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.587103 5033 raft_consensus.cc:729] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1074a5f7d64ca584665210f28e2d12, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.587431 5033 consensus_queue.cc:260] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.590832 5033 ts_tablet_manager.cc:1428] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:19:46.597929 5055 ts_tablet_manager.cc:1918] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:19:46.595867 5057 ts_tablet_manager.cc:1918] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:19:46.598304 5057 log.cc:1196] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/4eec0d3b325e4149beb0cc2ce689da99
I20260925 09:19:46.598989 5057 ts_tablet_manager.cc:1939] T 4eec0d3b325e4149beb0cc2ce689da99 P a97b1a609d2642a5a553647bc3723627: Deleting consensus metadata
I20260925 09:19:46.599195 5055 log.cc:1196] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/4eec0d3b325e4149beb0cc2ce689da99
I20260925 09:19:46.599783 5055 ts_tablet_manager.cc:1939] T 4eec0d3b325e4149beb0cc2ce689da99 P ff1074a5f7d64ca584665210f28e2d12: Deleting consensus metadata
I20260925 09:19:46.603068 4514 catalog_manager.cc:4841] TS a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801): tablet 4eec0d3b325e4149beb0cc2ce689da99 (table test-table [id=3b4d0642ff0f4ae1bae60593d4b2e629]) successfully deleted
I20260925 09:19:46.605098 4511 catalog_manager.cc:4841] TS ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265): tablet 4eec0d3b325e4149beb0cc2ce689da99 (table test-table [id=3b4d0642ff0f4ae1bae60593d4b2e629]) successfully deleted
I20260925 09:19:46.605387 5031 tablet_bootstrap.cc:492] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: No bootstrap required, opened a new log
I20260925 09:19:46.605535 5032 tablet_bootstrap.cc:492] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: No bootstrap required, opened a new log
I20260925 09:19:46.605584 5031 ts_tablet_manager.cc:1397] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Time spent bootstrapping tablet: real 0.036s user 0.012s sys 0.022s
I20260925 09:19:46.605732 5032 ts_tablet_manager.cc:1397] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.028s
I20260925 09:19:46.606667 5031 raft_consensus.cc:348] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.606660 5032 raft_consensus.cc:348] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.606933 5032 raft_consensus.cc:374] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.606941 5031 raft_consensus.cc:374] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.607049 5032 raft_consensus.cc:729] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97b1a609d2642a5a553647bc3723627, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.607059 5031 raft_consensus.cc:729] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb1efb2d58845b9b2e2a08734f10660, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.607416 5032 consensus_queue.cc:260] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.607391 5031 consensus_queue.cc:260] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.608160 5031 ts_tablet_manager.cc:1428] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 09:19:46.609400 5032 ts_tablet_manager.cc:1428] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260925 09:19:46.625552 4735 tablet.cc:2369] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:19:46.769436 5037 raft_consensus.cc:482] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:46.769731 5037 raft_consensus.cc:504] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.771728 5037 leader_election.cc:288] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1:46143), a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801)
I20260925 09:19:46.778717 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.772351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:42027 (local address 127.3.82.2:36801)
0925 09:19:46.772541 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.772548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.773282 (+ 734us) server_negotiation.cc:407] Connection header received
0925 09:19:46.773567 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.773573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.773684 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.773772 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.774258 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.774969 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.776784 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.777646 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.777789 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.777922 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.778261 (+ 339us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.778410 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:19:46.779049 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.771976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:36801 (local address 127.3.82.3:42027)
0925 09:19:46.773171 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.773195 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.773397 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.773782 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.773786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.773808 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.774089 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.774097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.775096 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.775105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.776567 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.776584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.777853 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.777860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.778065 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.778104 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.778210 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.778384 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":593,"thread_start_us":325,"threads_started":1}
I20260925 09:19:46.780056 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.771976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46143 (local address 127.3.82.3:48247)
0925 09:19:46.772538 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.772561 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.772718 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.773272 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.773278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.773309 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:46.773575 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.773586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.775168 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.775174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.777345 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.777362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.779191 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.779200 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.779382 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.779420 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.779521 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.779670 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"spinlock_wait_cycles":3840,"thread_start_us":244,"threads_started":1}
I20260925 09:19:46.780056 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.772598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:48247 (local address 127.3.82.1:46143)
0925 09:19:46.772883 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.772892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.772919 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:19:46.773007 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.773012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.773096 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.773224 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:46.773759 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.774973 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.777553 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.778869 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.779049 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.779176 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.779596 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.779744 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260925 09:19:46.779991 4823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd650989e904e4ab0d44aa573127ad8" candidate_uuid: "ff1074a5f7d64ca584665210f28e2d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a97b1a609d2642a5a553647bc3723627" is_pre_election: true
I20260925 09:19:46.780283 4823 raft_consensus.cc:2454] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1074a5f7d64ca584665210f28e2d12 in term 0.
I20260925 09:19:46.780872 4896 leader_election.cc:302] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a97b1a609d2642a5a553647bc3723627, ff1074a5f7d64ca584665210f28e2d12; no voters:
I20260925 09:19:46.781157 4684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd650989e904e4ab0d44aa573127ad8" candidate_uuid: "ff1074a5f7d64ca584665210f28e2d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb1efb2d58845b9b2e2a08734f10660" is_pre_election: true
I20260925 09:19:46.781381 5037 raft_consensus.cc:2781] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:46.781530 5037 raft_consensus.cc:482] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:46.781635 5037 raft_consensus.cc:3037] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:46.781735 4684 raft_consensus.cc:2454] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1074a5f7d64ca584665210f28e2d12 in term 0.
I20260925 09:19:46.785210 5037 raft_consensus.cc:504] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.786648 4684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd650989e904e4ab0d44aa573127ad8" candidate_uuid: "ff1074a5f7d64ca584665210f28e2d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3eb1efb2d58845b9b2e2a08734f10660"
I20260925 09:19:46.786691 4823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfd650989e904e4ab0d44aa573127ad8" candidate_uuid: "ff1074a5f7d64ca584665210f28e2d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a97b1a609d2642a5a553647bc3723627"
I20260925 09:19:46.786917 4684 raft_consensus.cc:3037] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:46.786926 4823 raft_consensus.cc:3037] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:46.789932 5037 leader_election.cc:288] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [CANDIDATE]: Term 1 election: Requested vote from peers 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1:46143), a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801)
I20260925 09:19:46.790480 4823 raft_consensus.cc:2454] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff1074a5f7d64ca584665210f28e2d12 in term 1.
I20260925 09:19:46.790488 4684 raft_consensus.cc:2454] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff1074a5f7d64ca584665210f28e2d12 in term 1.
I20260925 09:19:46.791109 4896 leader_election.cc:302] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3eb1efb2d58845b9b2e2a08734f10660, a97b1a609d2642a5a553647bc3723627, ff1074a5f7d64ca584665210f28e2d12; no voters:
I20260925 09:19:46.791523 5037 raft_consensus.cc:2781] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:46.792073 5037 raft_consensus.cc:686] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 1 LEADER]: Becoming Leader. State: Replica: ff1074a5f7d64ca584665210f28e2d12, State: Running, Role: LEADER
I20260925 09:19:46.792519 5037 consensus_queue.cc:237] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.796662 4525 catalog_manager.cc:5495] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "ff1074a5f7d64ca584665210f28e2d12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } health_report { overall_health: HEALTHY } } }
Sep 25 09:19:46 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327984, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:46.886015 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.868571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45195 (local address 127.0.0.1:51326)
0925 09:19:46.869561 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:46.869578 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:46.869716 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:46.870192 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:46.870196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:46.870852 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:46.871119 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.871129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.872708 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.872712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.873727 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:46.873740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.874784 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.874789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:46.874919 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.875682 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:46.875700 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:46.878368 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:46.882848 (+ 4480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:46.882863 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:46.882870 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:46.883613 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:46.884094 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:46.884097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:46.884100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:46.884335 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:46.885035 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:46.885042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:46.885214 (+ 172us) client_negotiation.cc:769] Sending connection context
0925 09:19:46.885361 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 09:19:46.885606 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":691,"threads_started":1}
I20260925 09:19:46.886322 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:46.868616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.3.82.62:45195)
0925 09:19:46.869323 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:46.869330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:46.869685 (+ 355us) server_negotiation.cc:407] Connection header received
0925 09:19:46.869864 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:46.869871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:46.869956 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:46.870095 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:46.871343 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.872530 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.873928 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:46.874605 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:46.874753 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:46.878676 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:46.878684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:46.878689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:46.878730 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:46.882501 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.883766 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.883775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.883779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.883957 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:46.884465 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:46.884470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:46.884474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:46.884886 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:46.885024 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:46.885519 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 09:19:46.885791 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":368,"threads_started":1}
I20260925 09:19:46.893112 4525 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:51326:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 09:19:46.894493 4525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:46.902386 4664 tablet_service.cc:1467] Processing CreateTablet for tablet b0556039346042689c5fc8f058cf9fc9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=700d935de02a43ce803a204a708e994e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:19:46.902841 4941 tablet_service.cc:1467] Processing CreateTablet for tablet b0556039346042689c5fc8f058cf9fc9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=700d935de02a43ce803a204a708e994e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:19:46.903095 4664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0556039346042689c5fc8f058cf9fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.903462 4941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0556039346042689c5fc8f058cf9fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.912045 5031 tablet_bootstrap.cc:492] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: Bootstrap starting.
I20260925 09:19:46.912277 5033 tablet_bootstrap.cc:492] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: Bootstrap starting.
I20260925 09:19:46.910434 4802 tablet_service.cc:1467] Processing CreateTablet for tablet b0556039346042689c5fc8f058cf9fc9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=700d935de02a43ce803a204a708e994e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:19:46.914079 4802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0556039346042689c5fc8f058cf9fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:46.914098 5033 tablet_bootstrap.cc:654] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.914413 5031 tablet_bootstrap.cc:654] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.924753 5031 tablet_bootstrap.cc:492] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: No bootstrap required, opened a new log
I20260925 09:19:46.925076 5031 ts_tablet_manager.cc:1397] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260925 09:19:46.926172 5032 tablet_bootstrap.cc:492] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: Bootstrap starting.
I20260925 09:19:46.926057 5031 raft_consensus.cc:348] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.926544 5031 raft_consensus.cc:374] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.926678 5031 raft_consensus.cc:729] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eb1efb2d58845b9b2e2a08734f10660, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.926993 5031 consensus_queue.cc:260] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.927546 5052 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:46.927872 5052 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:46.928022 5032 tablet_bootstrap.cc:654] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:46.929220 5031 ts_tablet_manager.cc:1428] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 09:19:46.931488 5032 tablet_bootstrap.cc:492] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: No bootstrap required, opened a new log
I20260925 09:19:46.931773 5032 ts_tablet_manager.cc:1397] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260925 09:19:46.932682 5032 raft_consensus.cc:348] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.932993 5032 raft_consensus.cc:374] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.933092 5032 raft_consensus.cc:729] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a97b1a609d2642a5a553647bc3723627, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.933372 5032 consensus_queue.cc:260] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.934003 5032 ts_tablet_manager.cc:1428] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 09:19:46.933871 5039 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:46.934253 5039 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:46.935999 5033 tablet_bootstrap.cc:492] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: No bootstrap required, opened a new log
I20260925 09:19:46.936190 5033 ts_tablet_manager.cc:1397] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.005s
I20260925 09:19:46.937166 5033 raft_consensus.cc:348] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.937423 5033 raft_consensus.cc:374] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:46.937508 5033 raft_consensus.cc:729] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1074a5f7d64ca584665210f28e2d12, State: Initialized, Role: FOLLOWER
I20260925 09:19:46.937770 5033 consensus_queue.cc:260] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:46.938381 5033 ts_tablet_manager.cc:1428] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:19:46.938282 5037 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:46.938577 5037 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:47.053695 5052 raft_consensus.cc:482] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:47.053979 5052 raft_consensus.cc:504] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:47.054703 5052 leader_election.cc:288] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801), ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265)
I20260925 09:19:47.055447 4823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0556039346042689c5fc8f058cf9fc9" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a97b1a609d2642a5a553647bc3723627" is_pre_election: true
I20260925 09:19:47.055722 4823 raft_consensus.cc:2454] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 0.
I20260925 09:19:47.055586 4962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0556039346042689c5fc8f058cf9fc9" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1074a5f7d64ca584665210f28e2d12" is_pre_election: true
I20260925 09:19:47.055945 4962 raft_consensus.cc:2454] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 0.
I20260925 09:19:47.056247 4617 leader_election.cc:302] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb1efb2d58845b9b2e2a08734f10660, a97b1a609d2642a5a553647bc3723627; no voters:
I20260925 09:19:47.056653 5052 raft_consensus.cc:2781] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:47.056789 5052 raft_consensus.cc:482] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:47.056860 5052 raft_consensus.cc:3037] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:47.059172 5052 raft_consensus.cc:504] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:47.059831 5052 leader_election.cc:288] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 election: Requested vote from peers a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801), ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265)
I20260925 09:19:47.060405 4823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0556039346042689c5fc8f058cf9fc9" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a97b1a609d2642a5a553647bc3723627"
I20260925 09:19:47.060663 4823 raft_consensus.cc:3037] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:47.060550 4962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0556039346042689c5fc8f058cf9fc9" candidate_uuid: "3eb1efb2d58845b9b2e2a08734f10660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1074a5f7d64ca584665210f28e2d12"
I20260925 09:19:47.060858 4962 raft_consensus.cc:3037] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:47.062879 4823 raft_consensus.cc:2454] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 1.
I20260925 09:19:47.063313 4962 raft_consensus.cc:2454] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3eb1efb2d58845b9b2e2a08734f10660 in term 1.
I20260925 09:19:47.063460 4617 leader_election.cc:302] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3eb1efb2d58845b9b2e2a08734f10660, a97b1a609d2642a5a553647bc3723627; no voters:
I20260925 09:19:47.063838 5052 raft_consensus.cc:2781] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:47.064224 5052 raft_consensus.cc:686] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 LEADER]: Becoming Leader. State: Replica: 3eb1efb2d58845b9b2e2a08734f10660, State: Running, Role: LEADER
I20260925 09:19:47.064546 5052 consensus_queue.cc:237] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } }
I20260925 09:19:47.066387 5075 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: TxnStatusTablet state changed. Reason: New leader 3eb1efb2d58845b9b2e2a08734f10660. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.066785 5075 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:19:47.067379 5077 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 09:19:47.067617 5077 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 09:19:47.068325 4526 catalog_manager.cc:5495] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } health_report { overall_health: UNKNOWN } } }
I20260925 09:19:47.086971 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.081514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46143 (local address 127.0.0.1:53666)
0925 09:19:47.081755 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:47.081775 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:47.081948 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:47.082523 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:47.082530 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:47.082543 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:47.082740 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:47.082755 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.083864 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.083870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:47.084890 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:47.084902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.085660 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.085666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:47.085899 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.085972 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.086394 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.086419 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:19:47.086504 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 09:19:47.086662 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 09:19:47.086939 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.081644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.3.82.1:46143)
0925 09:19:47.081847 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.081853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.081873 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:19:47.082245 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.082250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.082353 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.082482 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:47.082872 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.083730 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.085036 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.085485 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.085639 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.086153 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.086295 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.086393 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.086533 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.086621 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:19:47.091548 4962 raft_consensus.cc:1261] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Refusing update from remote peer 3eb1efb2d58845b9b2e2a08734f10660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:47.091962 4823 raft_consensus.cc:1261] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Refusing update from remote peer 3eb1efb2d58845b9b2e2a08734f10660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:47.092190 5075 consensus_queue.cc:1035] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:47.092593 5052 consensus_queue.cc:1035] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:47.105123 5039 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: TxnStatusTablet state changed. Reason: New leader 3eb1efb2d58845b9b2e2a08734f10660. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.105505 5039 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:47.107951 5079 mvcc.cc:205] Tried to move back new op lower bound from 7333183435120050176 to 7333183435019497472. Current Snapshot: MvccSnapshot[applied={T|T < 7333183435120050176}]
I20260925 09:19:47.109236 5052 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.109557 5052 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:19:47.115765 5037 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: TxnStatusTablet state changed. Reason: New leader 3eb1efb2d58845b9b2e2a08734f10660. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.116075 5037 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:47.119654 5075 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.119961 5075 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P 3eb1efb2d58845b9b2e2a08734f10660: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:19:47.126811 5080 mvcc.cc:205] Tried to move back new op lower bound from 7333183435120050176 to 7333183435019497472. Current Snapshot: MvccSnapshot[applied={T|T < 7333183435120050176}]
I20260925 09:19:47.127072 5039 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.127336 5039 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P a97b1a609d2642a5a553647bc3723627: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:47.129180 5037 tablet_replica.cc:439] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3eb1efb2d58845b9b2e2a08734f10660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 } } peers { permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 } } peers { permanent_uuid: "ff1074a5f7d64ca584665210f28e2d12" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36265 } } }
I20260925 09:19:47.129500 5037 tablet_replica.cc:442] T b0556039346042689c5fc8f058cf9fc9 P ff1074a5f7d64ca584665210f28e2d12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:47.144348 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.131293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.3.82.3:36265)
0925 09:19:47.134846 (+ 3553us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.134852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.135525 (+ 673us) server_negotiation.cc:407] Connection header received
0925 09:19:47.135837 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.135843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.135944 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.136128 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:47.136715 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.137941 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.140060 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.140786 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.140913 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.141721 (+ 808us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.143030 (+ 1309us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.143195 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.143846 (+ 651us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.143949 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3408,"thread_start_us":3278,"threads_started":1}
I20260925 09:19:47.168071 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.159355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:32789 (local address 127.3.82.1:46143)
0925 09:19:47.159642 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.159651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.162560 (+ 2909us) server_negotiation.cc:407] Connection header received
0925 09:19:47.162818 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.162822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.162898 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.163005 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:47.163636 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.164472 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.165995 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.166530 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.166691 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.167153 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.167306 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.167411 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.167557 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.167643 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:19:47.168102 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.159245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46143 (local address 127.3.82.3:32789)
0925 09:19:47.162417 (+ 3172us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:47.162438 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:47.162657 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:47.163027 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:47.163032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:47.163044 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:47.163368 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:47.163376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.164628 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.164634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:47.165775 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:47.165787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.166734 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.166743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:47.166887 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.166960 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.167413 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.167428 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 09:19:47.167533 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 09:19:47.167698 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2989,"thread_start_us":2820,"threads_started":1}
I20260925 09:19:47.179917 4823 raft_consensus.cc:1261] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Refusing update from remote peer ff1074a5f7d64ca584665210f28e2d12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:47.180202 4684 raft_consensus.cc:1261] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Refusing update from remote peer ff1074a5f7d64ca584665210f28e2d12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:47.180774 5063 consensus_queue.cc:1035] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a97b1a609d2642a5a553647bc3723627" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 36801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:47.181488 5037 consensus_queue.cc:1035] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eb1efb2d58845b9b2e2a08734f10660" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:47.244259 5081 mvcc.cc:205] Tried to move back new op lower bound from 7333183435481518080 to 7333183433907208192. Current Snapshot: MvccSnapshot[applied={T|T < 7333183435481518080}]
I20260925 09:19:47.275874 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.247205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:36265 (local address 127.3.82.1:38067)
0925 09:19:47.261367 (+ 14162us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:47.261390 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:47.261583 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:47.262664 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:47.262673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:47.262689 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:47.263201 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:47.263220 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.266041 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.266052 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:47.268515 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:47.268543 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.273698 (+ 5155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.273711 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:47.274020 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.274131 (+ 111us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.274805 (+ 674us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.274822 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 09:19:47.274918 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 09:19:47.275142 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13899,"thread_start_us":13679,"threads_started":1}
I20260925 09:19:47.276827 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.247772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:38067 (local address 127.3.82.3:36265)
0925 09:19:47.247966 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.247973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.261531 (+ 13558us) server_negotiation.cc:407] Connection header received
0925 09:19:47.262245 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.262251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.262347 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.262468 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:47.264353 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.265796 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.272181 (+ 6385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.273220 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.273422 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.274423 (+ 1001us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:47.274633 (+ 210us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:47.274725 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.276211 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.276336 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:19:47.348104 4525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:47.348423 4525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:47.352434 4525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e046fbed96f6441581cf3215389ec0c0: Sending DeleteTablet for 3 replicas of tablet dfd650989e904e4ab0d44aa573127ad8
I20260925 09:19:47.353683 4802 tablet_service.cc:1514] Processing DeleteTablet for tablet dfd650989e904e4ab0d44aa573127ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:47 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:47650
I20260925 09:19:47.353902 4941 tablet_service.cc:1514] Processing DeleteTablet for tablet dfd650989e904e4ab0d44aa573127ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:47 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:49778
I20260925 09:19:47.355123 4664 tablet_service.cc:1514] Processing DeleteTablet for tablet dfd650989e904e4ab0d44aa573127ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:47 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53648
I20260925 09:19:47.374559 5100 tablet_replica.cc:330] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: stopping tablet replica
I20260925 09:19:47.374850 5100 raft_consensus.cc:2229] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:47.375147 5100 raft_consensus.cc:2258] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:47.382867 5100 ts_tablet_manager.cc:1905] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:47.388957 5102 tablet_replica.cc:330] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: stopping tablet replica
I20260925 09:19:47.389191 5102 raft_consensus.cc:2229] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:47.391018 5102 raft_consensus.cc:2258] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:47.393816 5101 tablet_replica.cc:330] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: stopping tablet replica
I20260925 09:19:47.394142 5101 raft_consensus.cc:2229] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:19:47.394755 5101 raft_consensus.cc:2258] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:47.395246 5102 ts_tablet_manager.cc:1905] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:47.397461 5101 ts_tablet_manager.cc:1905] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:47.399672 5100 ts_tablet_manager.cc:1918] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:47.399850 5100 log.cc:1196] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/dfd650989e904e4ab0d44aa573127ad8
I20260925 09:19:47.400390 5100 ts_tablet_manager.cc:1939] T dfd650989e904e4ab0d44aa573127ad8 P a97b1a609d2642a5a553647bc3723627: Deleting consensus metadata
I20260925 09:19:47.401747 4514 catalog_manager.cc:4841] TS a97b1a609d2642a5a553647bc3723627 (127.3.82.2:36801): tablet dfd650989e904e4ab0d44aa573127ad8 (table test-table [id=7f2116ab74b74fc4aaa77f807ee08fb7]) successfully deleted
I20260925 09:19:47.410034 5101 ts_tablet_manager.cc:1918] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:47.410214 5101 log.cc:1196] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/dfd650989e904e4ab0d44aa573127ad8
I20260925 09:19:47.410707 5101 ts_tablet_manager.cc:1939] T dfd650989e904e4ab0d44aa573127ad8 P ff1074a5f7d64ca584665210f28e2d12: Deleting consensus metadata
I20260925 09:19:47.411938 4511 catalog_manager.cc:4841] TS ff1074a5f7d64ca584665210f28e2d12 (127.3.82.3:36265): tablet dfd650989e904e4ab0d44aa573127ad8 (table test-table [id=7f2116ab74b74fc4aaa77f807ee08fb7]) successfully deleted
I20260925 09:19:47.413619 5102 ts_tablet_manager.cc:1918] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:47.417971 5102 log.cc:1196] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/dfd650989e904e4ab0d44aa573127ad8
Sep 25 09:19:47 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:47.418651 5102 ts_tablet_manager.cc:1939] T dfd650989e904e4ab0d44aa573127ad8 P 3eb1efb2d58845b9b2e2a08734f10660: Deleting consensus metadata
I20260925 09:19:47.420075 4512 catalog_manager.cc:4841] TS 3eb1efb2d58845b9b2e2a08734f10660 (127.3.82.1:46143): tablet dfd650989e904e4ab0d44aa573127ad8 (table test-table [id=7f2116ab74b74fc4aaa77f807ee08fb7]) successfully deleted
I20260925 09:19:47.429523 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.388354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53678 (local address 127.3.82.1:46143)
0925 09:19:47.388614 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.388622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.402717 (+ 14095us) server_negotiation.cc:407] Connection header received
0925 09:19:47.402991 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.402998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.403120 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.403262 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:47.406203 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.410381 (+ 4178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.412964 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.413647 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.413831 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.420541 (+ 6710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:47.420555 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:47.420589 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:47.420647 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:47.425206 (+ 4559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.426419 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.426430 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.426434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.426665 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.427260 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.427265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.427268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.427763 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:47.428172 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.428820 (+ 648us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.429111 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260925 09:19:47.431600 4729 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53678
I20260925 09:19:47.476706 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.447832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.3.82.62:45195)
0925 09:19:47.456433 (+ 8601us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.456442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.456479 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:19:47.456581 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.456587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.456701 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.456864 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:47.459120 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.460479 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.462655 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.463380 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.463535 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.467404 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:47.467413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:47.467418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:47.467501 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:47.471876 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.473022 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.473030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.473034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.473233 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.473778 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.473785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.473789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.474251 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:47.475804 (+ 1553us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.475997 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.476255 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8443,"thread_start_us":8287,"threads_started":1}
W20260925 09:19:47.477473 4525 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51328
I20260925 09:19:47.480176 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4598
I20260925 09:19:47.507925 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.482432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.3.82.1:46143)
0925 09:19:47.483137 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.483148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.488421 (+ 5273us) server_negotiation.cc:407] Connection header received
0925 09:19:47.488741 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.488749 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.488852 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.488939 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:47.491409 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.492705 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.494759 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.495560 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.495683 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.499458 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:47.499468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:47.499473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:47.499528 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:47.503864 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.504937 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.504944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.504949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.505183 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.505761 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.505768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.505772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.506201 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:47.506506 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.507149 (+ 643us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.507438 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501}
W20260925 09:19:47.509779 4729 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53684
W20260925 09:19:47.510608 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:19:47.510869 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4598
I20260925 09:19:47.544452 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4738
Sep 25 09:19:47 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:19:47.572871 5109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.545474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.3.82.2:36801)
0925 09:19:47.550360 (+ 4886us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.550367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.550393 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:19:47.550471 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.550478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.550579 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.550727 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:47.552666 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.553868 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.556256 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.557011 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.557161 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.563546 (+ 6385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:47.563573 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:47.563601 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:47.563667 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:47.568768 (+ 5101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.569939 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.569953 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.569959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.570187 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.570714 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.570722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.570728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.571299 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:47.571651 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.572020 (+ 369us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.572334 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4743,"thread_start_us":4624,"threads_started":1}
W20260925 09:19:47.573729 4868 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47652
W20260925 09:19:47.574566 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4738: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:19:47.574790 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4738
I20260925 09:19:47.598590 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4877
Sep 25 09:19:47 dist-test-slave-4wm5 krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327986, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:19:47.622557 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:47.599545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.3.82.3:36265)
0925 09:19:47.599753 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:47.599760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:47.599976 (+ 216us) server_negotiation.cc:407] Connection header received
0925 09:19:47.600193 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:47.600200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:47.600326 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:47.600461 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:47.602400 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.603592 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.605929 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:47.606653 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:47.606805 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:47.613542 (+ 6737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:47.613575 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:47.613606 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:47.613686 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:47.618974 (+ 5288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.620051 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.620061 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.620077 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.620255 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:47.620733 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:47.620741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:47.620747 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:47.621243 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:47.621571 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:47.621958 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:19:47.622205 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 09:19:47.623317 5007 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49798
W20260925 09:19:47.624089 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4877: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:19:47.624250 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4877
I20260925 09:19:47.639746 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 4495
I20260925 09:19:47.641084 4567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:47.758560 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 4495
2026-09-25T09:19:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5103 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:19:47 dist-test-slave-4wm5 krb5kdc[5113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:19:47 dist-test-slave-4wm5 krb5kdc[5113](info): set up 2 sockets
Sep 25 09:19:47 dist-test-slave-4wm5 krb5kdc[5113](info): commencing operation
krb5kdc: starting...
W20260925 09:19:48.882419 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.005s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:19:49 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:19:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:19:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:19:49.095028 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36081
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:49.436760 5126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:49.437160 5126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:49.437332 5126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:49.445677 5126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:19:49.445919 5126 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 09:19:49.446059 5126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:49.446180 5126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:49.446270 5126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:19:49.446362 5126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:19:49.457401 5126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36081
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:49.458891 5126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:49.460398 5126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:49.473682 5131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:49.474197 5132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:49.474362 5134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:49.475488 5126 server_base.cc:1034] running on GCE node
I20260925 09:19:49.476483 5126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:49.478226 5126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:49.479517 5126 hybrid_clock.cc:648] HybridClock initialized: now 1790327989479456 us; error 41 us; skew 500 ppm
I20260925 09:19:49.479919 5126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:49 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327989, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:49.483741 5126 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:19:49.485600 5126 webserver.cc:460] Webserver started at http://127.3.82.62:36103/ using document root <none> and password file <none>
I20260925 09:19:49.486446 5126 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:49.486595 5126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:49.486948 5126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:49.490284 5126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "4887fa3c2e104c5583bc702051bef7cb"
format_stamp: "Formatted at 2026-09-25 09:19:49 on dist-test-slave-4wm5"
server_key: "6ab1e404a818b3896ad5671aed221489"
server_key_iv: "d5cc3a37db998407e36002760c1322bc"
server_key_version: "encryptionkey@0"
I20260925 09:19:49.491185 5126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "4887fa3c2e104c5583bc702051bef7cb"
format_stamp: "Formatted at 2026-09-25 09:19:49 on dist-test-slave-4wm5"
server_key: "6ab1e404a818b3896ad5671aed221489"
server_key_iv: "d5cc3a37db998407e36002760c1322bc"
server_key_version: "encryptionkey@0"
I20260925 09:19:49.496960 5126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:19:49.500984 5141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:49.502674 5126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:19:49.502893 5126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "4887fa3c2e104c5583bc702051bef7cb"
format_stamp: "Formatted at 2026-09-25 09:19:49 on dist-test-slave-4wm5"
server_key: "6ab1e404a818b3896ad5671aed221489"
server_key_iv: "d5cc3a37db998407e36002760c1322bc"
server_key_version: "encryptionkey@0"
I20260925 09:19:49.503150 5126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:49.521394 5126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:49.526410 5126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:49.526738 5126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:49.549948 5126 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36081
I20260925 09:19:49.550004 5202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36081 every 8 connection(s)
I20260925 09:19:49.551371 5126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:19:49.555770 5203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:49.556154 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 5126
I20260925 09:19:49.556375 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:19:49.557332 3400 external_mini_cluster.cc:1428] Setting key 409bce2e823299a340ff4d30c7083ea3
I20260925 09:19:49.572780 5203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Bootstrap starting.
Sep 25 09:19:49 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:49.576741 5203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:49.578384 5203 log.cc:824] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:49.582054 5203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: No bootstrap required, opened a new log
I20260925 09:19:49.585155 5206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:49.559661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.3.82.62:36081)
0925 09:19:49.560636 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:49.560649 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:49.560694 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:19:49.563040 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:49.563066 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:49.563529 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:49.563924 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:49.566244 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:49.567609 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:49.569685 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:49.570668 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:49.570846 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:49.576032 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:49.576042 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:49.576059 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:49.576109 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:49.580081 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:49.580916 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:49.580924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:49.580934 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:49.581190 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:49.581611 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:49.581618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:49.581623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:49.582439 (+ 816us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:49.582781 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:49.583187 (+ 406us) server_negotiation.cc:299] Negotiation successful
0925 09:19:49.583664 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":534,"threads_started":1}
I20260925 09:19:49.591532 5203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4887fa3c2e104c5583bc702051bef7cb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36081 } }
I20260925 09:19:49.591944 5203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:49.592053 5203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4887fa3c2e104c5583bc702051bef7cb, State: Initialized, Role: FOLLOWER
I20260925 09:19:49.592805 5203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4887fa3c2e104c5583bc702051bef7cb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36081 } }
I20260925 09:19:49.593075 5203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:19:49.593220 5203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:19:49.593403 5203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:49.595714 5203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4887fa3c2e104c5583bc702051bef7cb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36081 } }
I20260925 09:19:49.596329 5203 leader_election.cc:302] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4887fa3c2e104c5583bc702051bef7cb; no voters:
I20260925 09:19:49.597045 5203 leader_election.cc:288] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:19:49.597146 5208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:49.597944 5208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [term 1 LEADER]: Becoming Leader. State: Replica: 4887fa3c2e104c5583bc702051bef7cb, State: Running, Role: LEADER
I20260925 09:19:49.598730 5208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4887fa3c2e104c5583bc702051bef7cb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36081 } }
I20260925 09:19:49.599766 5203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:19:49.602263 5210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4887fa3c2e104c5583bc702051bef7cb. Latest consensus state: current_term: 1 leader_uuid: "4887fa3c2e104c5583bc702051bef7cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4887fa3c2e104c5583bc702051bef7cb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36081 } } }
I20260925 09:19:49.602252 5209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4887fa3c2e104c5583bc702051bef7cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4887fa3c2e104c5583bc702051bef7cb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36081 } } }
I20260925 09:19:49.602777 5209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:49.605188 5210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:49.611063 5213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:19:49.617066 5213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:19:49.626925 5213 catalog_manager.cc:1347] Generated new cluster ID: 1d4f9bfdf2c144ec9d89f034f823f207
I20260925 09:19:49.627080 5213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:19:49.642663 5213 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:19:49.644207 5213 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:19:49.653533 5213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Generated new TSK 0
I20260925 09:19:49.654503 5213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:19:49.710116 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36081
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:50.080104 5229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:50.080444 5229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:50.080613 5229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:50.088255 5229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:50.088382 5229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:50.088528 5229 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 09:19:50.088606 5229 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:19:50.088744 5229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:19:50.099220 5229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:36081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:50.100682 5229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:50.102154 5229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:50.118140 5237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:50.118108 5235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:50.118242 5234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:50.119220 5229 server_base.cc:1034] running on GCE node
I20260925 09:19:50.119940 5229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:50.121083 5229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:50.122318 5229 hybrid_clock.cc:648] HybridClock initialized: now 1790327990122273 us; error 41 us; skew 500 ppm
I20260925 09:19:50.122653 5229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:50 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327990, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:50.130633 5229 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:19:50.132421 5229 webserver.cc:460] Webserver started at http://127.3.82.1:43609/ using document root <none> and password file <none>
I20260925 09:19:50.133277 5229 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:50.133443 5229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:50.133792 5229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:50.137147 5229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "661ee9937945416ab5760db988166a42"
format_stamp: "Formatted at 2026-09-25 09:19:50 on dist-test-slave-4wm5"
server_key: "0cff8cda803dcab9dd47fa7b6825408b"
server_key_iv: "45dc6c4743a9d01153e8f45ee591cedc"
server_key_version: "encryptionkey@0"
I20260925 09:19:50.138047 5229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "661ee9937945416ab5760db988166a42"
format_stamp: "Formatted at 2026-09-25 09:19:50 on dist-test-slave-4wm5"
server_key: "0cff8cda803dcab9dd47fa7b6825408b"
server_key_iv: "45dc6c4743a9d01153e8f45ee591cedc"
server_key_version: "encryptionkey@0"
I20260925 09:19:50.144230 5229 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:19:50.148319 5244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:50.150244 5229 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260925 09:19:50.150460 5229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "661ee9937945416ab5760db988166a42"
format_stamp: "Formatted at 2026-09-25 09:19:50 on dist-test-slave-4wm5"
server_key: "0cff8cda803dcab9dd47fa7b6825408b"
server_key_iv: "45dc6c4743a9d01153e8f45ee591cedc"
server_key_version: "encryptionkey@0"
I20260925 09:19:50.150743 5229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:50.209012 5229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:50.214771 5229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:50.215124 5229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:50.218143 5229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:50.218252 5229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:50.219038 5229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:50.219182 5229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:19:50 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327990, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:50.246671 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.219209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:38363 (local address 127.3.82.62:36081)
0925 09:19:50.220134 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:50.220144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:50.221029 (+ 885us) server_negotiation.cc:407] Connection header received
0925 09:19:50.223467 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:50.223477 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:50.223611 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:50.223779 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:50.226562 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.228097 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.231120 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.231994 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.232233 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.237673 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:50.237678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:50.237682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:50.237720 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:50.240832 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.242321 (+ 1489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.242335 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.242337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.242480 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.243166 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.243174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.243180 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.243612 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:50.243794 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:50.246061 (+ 2267us) server_negotiation.cc:299] Negotiation successful
0925 09:19:50.246312 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":633,"threads_started":1}
I20260925 09:19:50.247126 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.219729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.3.82.1:38363)
0925 09:19:50.220803 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:50.220858 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:50.223052 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:50.224114 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:50.224127 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:50.225021 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:50.226220 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.226263 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.228319 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.228328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.230892 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.230919 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.232369 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.232382 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.232611 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.233664 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:50.233698 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:50.237347 (+ 3649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:50.241063 (+ 3716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.241083 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.241114 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.242064 (+ 950us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.242675 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.242683 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.242689 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.243024 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.243769 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:50.243782 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:50.244228 (+ 446us) client_negotiation.cc:769] Sending connection context
0925 09:19:50.244661 (+ 433us) client_negotiation.cc:240] Negotiation successful
0925 09:19:50.245242 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":469,"threads_started":1}
I20260925 09:19:50.267851 5229 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:39727
I20260925 09:19:50.267948 5364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:39727 every 8 connection(s)
I20260925 09:19:50.269490 5229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:19:50.271823 5365 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36081
I20260925 09:19:50.272065 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.256450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.3.82.1:40959)
0925 09:19:50.257245 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:50.257271 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:50.257480 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:50.257911 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:50.257916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:50.258550 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:50.259193 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.259206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.260714 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.260720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.262364 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.262381 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.263197 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.263204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.263401 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.264314 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:50.264332 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:50.265518 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:50.268984 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.268989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.268993 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.269392 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.269864 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.269871 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.269877 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.270060 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.270939 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:50.270949 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:50.271222 (+ 273us) client_negotiation.cc:769] Sending connection context
0925 09:19:50.271347 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 09:19:50.271620 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":366,"threads_started":1}
I20260925 09:19:50.272085 5365 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:50.272845 5365 heartbeater.cc:511] Master 127.3.82.62:36081 requested a full tablet report, sending...
I20260925 09:19:50.273161 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.256332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:40959 (local address 127.3.82.62:36081)
0925 09:19:50.256521 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:50.256525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:50.257392 (+ 867us) server_negotiation.cc:407] Connection header received
0925 09:19:50.257645 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:50.257651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:50.257739 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:50.257876 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:50.259438 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.260438 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.262550 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.262990 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.263132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.265744 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:50.265750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:50.265753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:50.265786 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:50.268842 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.269536 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.269549 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.269555 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.269731 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.270227 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.270235 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.270238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.270729 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:50.272503 (+ 1774us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:50.272620 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 09:19:50.272811 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:19:50.275463 5158 ts_manager.cc:194] Registered new tserver with Master: 661ee9937945416ab5760db988166a42 (127.3.82.1:39727)
I20260925 09:19:50.275890 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 5229
I20260925 09:19:50.276111 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:19:50.276902 3400 external_mini_cluster.cc:1428] Setting key 26d5a6f0aa17e093f76dd051420f6aa1
I20260925 09:19:50.277532 5158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:38363
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:19:50.322935 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36081
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:50.665436 5369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:50.665752 5369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:50.665928 5369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:50.673358 5369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:50.673475 5369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:50.673561 5369 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 09:19:50.673614 5369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:19:50.673677 5369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:19:50.683799 5369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:36081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:50.685103 5369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:50.686542 5369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:50.700332 5374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:50.700398 5375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:50.700376 5377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:50.701417 5369 server_base.cc:1034] running on GCE node
I20260925 09:19:50.702234 5369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:50.703279 5369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:50.704525 5369 hybrid_clock.cc:648] HybridClock initialized: now 1790327990704436 us; error 72 us; skew 500 ppm
I20260925 09:19:50.704870 5369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:50 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327990, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:50.708310 5369 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:19:50.710036 5369 webserver.cc:460] Webserver started at http://127.3.82.2:39649/ using document root <none> and password file <none>
I20260925 09:19:50.710726 5369 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:50.710830 5369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:50.711128 5369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:50.714258 5369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "8358935773044bb49c0de5c30adbde61"
format_stamp: "Formatted at 2026-09-25 09:19:50 on dist-test-slave-4wm5"
server_key: "e36123ced67c308aa8973f891f32a8d6"
server_key_iv: "a34f2406a1e2f9c9c453afce8b5c6099"
server_key_version: "encryptionkey@0"
I20260925 09:19:50.714980 5369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "8358935773044bb49c0de5c30adbde61"
format_stamp: "Formatted at 2026-09-25 09:19:50 on dist-test-slave-4wm5"
server_key: "e36123ced67c308aa8973f891f32a8d6"
server_key_iv: "a34f2406a1e2f9c9c453afce8b5c6099"
server_key_version: "encryptionkey@0"
I20260925 09:19:50.720709 5369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:19:50.724819 5384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:50.726569 5369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:19:50.726784 5369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "8358935773044bb49c0de5c30adbde61"
format_stamp: "Formatted at 2026-09-25 09:19:50 on dist-test-slave-4wm5"
server_key: "e36123ced67c308aa8973f891f32a8d6"
server_key_iv: "a34f2406a1e2f9c9c453afce8b5c6099"
server_key_version: "encryptionkey@0"
I20260925 09:19:50.727135 5369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:50.741477 5369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:50.746706 5369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:50.747056 5369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:50.750208 5369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:50.750309 5369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:50.750456 5369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:50.750568 5369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:19:50 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327990, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:50.781292 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.751535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:36913 (local address 127.3.82.62:36081)
0925 09:19:50.751764 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:50.751771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:50.753495 (+ 1724us) server_negotiation.cc:407] Connection header received
0925 09:19:50.757446 (+ 3951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:50.757456 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:50.757578 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:50.757706 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:50.761077 (+ 3371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.762200 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.765579 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.766135 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.766340 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.772659 (+ 6319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:50.772668 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:50.772674 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:50.772745 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:50.776325 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.777544 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.777552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.777557 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.777764 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.779079 (+ 1315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.779086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.779091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.779550 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:50.779736 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:50.780626 (+ 890us) server_negotiation.cc:299] Negotiation successful
0925 09:19:50.780902 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 09:19:50.783057 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.752188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.3.82.2:36913)
0925 09:19:50.753253 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:50.753318 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:50.756950 (+ 3632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:50.758077 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:50.758096 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:50.759337 (+ 1241us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:50.760740 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.760787 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.762365 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.762371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.765318 (+ 2947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.765346 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.766457 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.766465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.766723 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.767596 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:50.767643 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:50.772294 (+ 4651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:50.776525 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.776544 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.776573 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.777304 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.778502 (+ 1198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.778512 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.778517 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.778914 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.779729 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:50.779738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:50.780178 (+ 440us) client_negotiation.cc:769] Sending connection context
0925 09:19:50.780513 (+ 335us) client_negotiation.cc:240] Negotiation successful
0925 09:19:50.781184 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":450,"threads_started":1}
I20260925 09:19:50.798743 5369 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:42399
I20260925 09:19:50.798786 5503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:42399 every 8 connection(s)
I20260925 09:19:50.800211 5369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:19:50.802955 5504 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36081
I20260925 09:19:50.803213 5504 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:50.803928 5504 heartbeater.cc:511] Master 127.3.82.62:36081 requested a full tablet report, sending...
I20260925 09:19:50.804654 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 5369
I20260925 09:19:50.804847 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:19:50.805465 3400 external_mini_cluster.cc:1428] Setting key c94b09e4fc561aa082bd15a3351882fc
I20260925 09:19:50.806615 5157 ts_manager.cc:194] Registered new tserver with Master: 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399)
I20260925 09:19:50.808024 5157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:36913
I20260925 09:19:50.809875 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.792518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:45281 (local address 127.3.82.62:36081)
0925 09:19:50.792793 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:50.792803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:50.793669 (+ 866us) server_negotiation.cc:407] Connection header received
0925 09:19:50.794067 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:50.794077 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:50.794189 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:50.794278 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:50.796079 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.797128 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.799087 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.799533 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.799699 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.802298 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:50.802307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:50.802313 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:50.802361 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:50.806232 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.807341 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.807345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.807348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.807499 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:50.807962 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:50.807967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:50.807971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:50.808581 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:50.808739 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:50.809219 (+ 480us) server_negotiation.cc:299] Negotiation successful
0925 09:19:50.809455 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 09:19:50.810024 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:50.792549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.3.82.2:45281)
0925 09:19:50.793522 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:50.793558 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:50.793762 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:50.794528 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:50.794535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:50.795307 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:50.795903 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.795916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.797297 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.797312 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.798876 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:50.798895 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:50.799776 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:50.799786 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:50.800088 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:50.800771 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:50.800793 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:50.802024 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:50.806607 (+ 4583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.806616 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.806620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.807188 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.807651 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:50.807655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:50.807657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:50.807830 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:50.808720 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:50.808726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:50.808935 (+ 209us) client_negotiation.cc:769] Sending connection context
0925 09:19:50.809110 (+ 175us) client_negotiation.cc:240] Negotiation successful
0925 09:19:50.809491 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":546,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:19:50.854604 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36081
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:51.207113 5508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:51.207410 5508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:51.207513 5508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:51.215399 5508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:51.215525 5508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:51.215619 5508 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 09:19:51.215673 5508 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:19:51.215734 5508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:19:51.226114 5508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:36081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:51.227432 5508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:51.228781 5508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:51.244047 5513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:51.244189 5514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:51.244199 5516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:51.245075 5508 server_base.cc:1034] running on GCE node
I20260925 09:19:51.245955 5508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:51.247090 5508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:51.248394 5508 hybrid_clock.cc:648] HybridClock initialized: now 1790327991248314 us; error 68 us; skew 500 ppm
I20260925 09:19:51.248801 5508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:51 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327991, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:51.253072 5508 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:19:51.254988 5508 webserver.cc:460] Webserver started at http://127.3.82.3:43737/ using document root <none> and password file <none>
I20260925 09:19:51.255952 5508 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:51.256117 5508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:51.256523 5508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:51.259936 5508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "4c25bdf951d74e0f81505e97284f9871"
format_stamp: "Formatted at 2026-09-25 09:19:51 on dist-test-slave-4wm5"
server_key: "b14f77f0ab5ccc4543af727ece337061"
server_key_iv: "64eebce63c16e954f8955e9d25fdfcfb"
server_key_version: "encryptionkey@0"
I20260925 09:19:51.260874 5508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "4c25bdf951d74e0f81505e97284f9871"
format_stamp: "Formatted at 2026-09-25 09:19:51 on dist-test-slave-4wm5"
server_key: "b14f77f0ab5ccc4543af727ece337061"
server_key_iv: "64eebce63c16e954f8955e9d25fdfcfb"
server_key_version: "encryptionkey@0"
I20260925 09:19:51.266902 5508 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260925 09:19:51.271817 5523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:51.273449 5508 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:19:51.273627 5508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "4c25bdf951d74e0f81505e97284f9871"
format_stamp: "Formatted at 2026-09-25 09:19:51 on dist-test-slave-4wm5"
server_key: "b14f77f0ab5ccc4543af727ece337061"
server_key_iv: "64eebce63c16e954f8955e9d25fdfcfb"
server_key_version: "encryptionkey@0"
I20260925 09:19:51.273885 5508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:51.280682 5365 heartbeater.cc:503] Master 127.3.82.62:36081 was elected leader, sending a full tablet report...
I20260925 09:19:51.290756 5508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:51.295394 5508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:51.295692 5508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:51.298599 5508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:51.298759 5508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:51.298892 5508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:51.298990 5508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:19:51 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327991, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:51.323356 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.299446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:39813 (local address 127.3.82.62:36081)
0925 09:19:51.299720 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.299727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.301234 (+ 1507us) server_negotiation.cc:407] Connection header received
0925 09:19:51.303575 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.303581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.303699 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.303856 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:51.307015 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.308158 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.310036 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.310692 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.310840 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.315573 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:51.315579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:51.315582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:51.315620 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:51.319257 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.320353 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.320362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.320367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.320561 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.321304 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.321312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.321318 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.321718 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:51.321940 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.322708 (+ 768us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.322958 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260925 09:19:51.324545 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.300004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.3.82.3:39813)
0925 09:19:51.301026 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.301081 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.303238 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.304226 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.304240 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.305276 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:51.306653 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.306691 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.308335 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.308342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.309801 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.309815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.310956 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.310965 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.311268 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.312207 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:51.312300 (+ 93us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:51.315269 (+ 2969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:51.319429 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:51.319440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:51.319468 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:51.320197 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:51.320732 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:51.320741 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:51.320747 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:51.321124 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:51.321891 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:51.321900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:51.322305 (+ 405us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.322600 (+ 295us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.323268 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":400,"threads_started":1}
I20260925 09:19:51.351200 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.332064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50527 (local address 127.3.82.62:36081)
0925 09:19:51.332355 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.332365 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.333062 (+ 697us) server_negotiation.cc:407] Connection header received
0925 09:19:51.333292 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.333297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.333396 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.333521 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:51.335066 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.336128 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.338346 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.339312 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.339523 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.341985 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:51.341997 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:51.342007 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:51.342067 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:51.347172 (+ 5105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.348310 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.348317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.348323 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.348512 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.349045 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.349054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.349061 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.349636 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:51.349954 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.350518 (+ 564us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.350793 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 09:19:51.351294 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.331912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.3.82.3:50527)
0925 09:19:51.332938 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.332955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.333147 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.333556 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.333561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.334201 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:51.334798 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.334822 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.336345 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.336351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.338114 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.338140 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.339629 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.339637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.339860 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.340703 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:51.340762 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:51.341605 (+ 843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:51.347403 (+ 5798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:51.347415 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:51.347422 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:51.348085 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:51.348645 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:51.348661 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:51.348663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:51.348878 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:51.349836 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:51.349842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:51.350156 (+ 314us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.350359 (+ 203us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.350801 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":660,"threads_started":1}
I20260925 09:19:51.364182 5508 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:38347
I20260925 09:19:51.364217 5642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:38347 every 8 connection(s)
I20260925 09:19:51.365696 5508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:19:51.367244 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 5508
I20260925 09:19:51.367451 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:19:51.368049 3400 external_mini_cluster.cc:1428] Setting key 9b655dda8176e66f69855854e4195a4b
I20260925 09:19:51.368386 5643 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36081
I20260925 09:19:51.368657 5643 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:51.369619 5643 heartbeater.cc:511] Master 127.3.82.62:36081 requested a full tablet report, sending...
I20260925 09:19:51.371380 5158 ts_manager.cc:194] Registered new tserver with Master: 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:51.372319 5158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:39813
I20260925 09:19:51.384660 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:19:51 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:19:51 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:51.437668 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.421306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.3.82.62:36081)
0925 09:19:51.421507 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.421511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.421888 (+ 377us) server_negotiation.cc:407] Connection header received
0925 09:19:51.422221 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.422226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.422296 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.422400 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:51.424003 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.424828 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.426231 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.426675 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.426802 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.431462 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:51.431468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:51.431472 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:51.431516 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:51.434709 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.435458 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.435463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.435465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.435610 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.436056 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.436060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.436062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.436356 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:51.436617 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.436969 (+ 352us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.437308 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:19:51.442751 5158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:19:51.450413 5158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:51.489171 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.465515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39727 (local address 127.0.0.1:43784)
0925 09:19:51.466418 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.466459 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.466655 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.480521 (+ 13866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.480534 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.480586 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:51.481196 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.481215 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.483612 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.483617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.486137 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.486150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.488110 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.488117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.488369 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.488440 (+ 71us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.488552 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.488761 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":699,"threads_started":1}
I20260925 09:19:51.490283 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.466079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.3.82.1:39727)
0925 09:19:51.479263 (+ 13184us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.479278 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.479327 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:19:51.479490 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.479503 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.479915 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.481488 (+ 1573us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:51.481753 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.483455 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.486316 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.487884 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.489370 (+ 1486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.489609 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.489698 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.489933 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12910,"thread_start_us":14053,"threads_started":1}
I20260925 09:19:51.490597 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.465495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42399 (local address 127.0.0.1:53298)
0925 09:19:51.479263 (+ 13768us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.479297 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.479566 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.482742 (+ 3176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.482747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.482778 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:51.483152 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.483162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.485679 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.485685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.487701 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.487715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.489757 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.489765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.489959 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.489991 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.490094 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.490237 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13472,"thread_start_us":14159,"threads_started":1}
I20260925 09:19:51.490869 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.466101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.3.82.2:42399)
0925 09:19:51.481886 (+ 15785us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.481898 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.481933 (+ 35us) server_negotiation.cc:407] Connection header received
0925 09:19:51.482046 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.482057 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.482339 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.482606 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:51.484023 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.485514 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.487915 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.489272 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.489396 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.489613 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.490182 (+ 569us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.490579 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15562,"thread_start_us":15427,"threads_started":1}
I20260925 09:19:51.493031 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.467144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38347 (local address 127.0.0.1:45284)
0925 09:19:51.482780 (+ 15636us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.482803 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.483119 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.484255 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.484260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.484287 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:51.484793 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.484805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.487166 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.487172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.489414 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.489429 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.491984 (+ 2555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.491991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.492354 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.492390 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.492493 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.492649 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15482,"mutex_wait_us":1,"thread_start_us":15327,"threads_started":1}
I20260925 09:19:51.493882 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.467869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.3.82.3:38347)
0925 09:19:51.482313 (+ 14444us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.482322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.483004 (+ 682us) server_negotiation.cc:407] Connection header received
0925 09:19:51.483379 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.483388 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.483677 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.483880 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:51.485223 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.486991 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.489621 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.491755 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.491893 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.492130 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.493346 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.493520 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14277,"thread_start_us":14089,"threads_started":1}
I20260925 09:19:51.494707 5434 tablet_service.cc:1467] Processing CreateTablet for tablet 45e198c2e639403d90b2f14dd64775c4 (DEFAULT_TABLE table=test-table [id=57b00fc546fa45b2b35ba8751f820348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:51.494678 5295 tablet_service.cc:1467] Processing CreateTablet for tablet 45e198c2e639403d90b2f14dd64775c4 (DEFAULT_TABLE table=test-table [id=57b00fc546fa45b2b35ba8751f820348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:51.496136 5434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e198c2e639403d90b2f14dd64775c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.496165 5295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e198c2e639403d90b2f14dd64775c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.497772 5577 tablet_service.cc:1467] Processing CreateTablet for tablet 45e198c2e639403d90b2f14dd64775c4 (DEFAULT_TABLE table=test-table [id=57b00fc546fa45b2b35ba8751f820348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:51.499776 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e198c2e639403d90b2f14dd64775c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.508993 5662 tablet_bootstrap.cc:492] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Bootstrap starting.
I20260925 09:19:51.511253 5663 tablet_bootstrap.cc:492] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Bootstrap starting.
I20260925 09:19:51.514425 5662 tablet_bootstrap.cc:654] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.515532 5663 tablet_bootstrap.cc:654] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.515786 5662 log.cc:824] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:51.516186 5664 tablet_bootstrap.cc:492] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Bootstrap starting.
I20260925 09:19:51.517037 5663 log.cc:824] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:51.519793 5662 tablet_bootstrap.cc:492] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: No bootstrap required, opened a new log
I20260925 09:19:51.520097 5662 ts_tablet_manager.cc:1397] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260925 09:19:51.520216 5663 tablet_bootstrap.cc:492] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: No bootstrap required, opened a new log
I20260925 09:19:51.520426 5664 tablet_bootstrap.cc:654] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.520495 5663 ts_tablet_manager.cc:1397] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260925 09:19:51.522153 5664 log.cc:824] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:51.525791 5664 tablet_bootstrap.cc:492] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: No bootstrap required, opened a new log
I20260925 09:19:51.526181 5664 ts_tablet_manager.cc:1397] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260925 09:19:51.529182 5662 raft_consensus.cc:348] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.529598 5662 raft_consensus.cc:374] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.529711 5662 raft_consensus.cc:729] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.530457 5662 consensus_queue.cc:260] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.531694 5504 heartbeater.cc:503] Master 127.3.82.62:36081 was elected leader, sending a full tablet report...
I20260925 09:19:51.531375 5663 raft_consensus.cc:348] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.531910 5663 raft_consensus.cc:374] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.532004 5663 raft_consensus.cc:729] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee9937945416ab5760db988166a42, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.532554 5662 ts_tablet_manager.cc:1428] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Time spent starting tablet: real 0.012s user 0.014s sys 0.000s
I20260925 09:19:51.532653 5663 consensus_queue.cc:260] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.534633 5663 ts_tablet_manager.cc:1428] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260925 09:19:51.537408 5664 raft_consensus.cc:348] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.537935 5664 raft_consensus.cc:374] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.538067 5664 raft_consensus.cc:729] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.538743 5664 consensus_queue.cc:260] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.539927 5643 heartbeater.cc:503] Master 127.3.82.62:36081 was elected leader, sending a full tablet report...
I20260925 09:19:51.541783 5664 ts_tablet_manager.cc:1428] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Time spent starting tablet: real 0.015s user 0.007s sys 0.006s
W20260925 09:19:51.553445 5505 tablet.cc:2369] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:19:51.556440 5670 raft_consensus.cc:482] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:51.556799 5670 raft_consensus.cc:504] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.558995 5670 leader_election.cc:288] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399), 661ee9937945416ab5760db988166a42 (127.3.82.1:39727)
I20260925 09:19:51.566182 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.559242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39727 (local address 127.3.82.3:35931)
0925 09:19:51.559507 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.559530 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.559679 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.560386 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.560392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.560418 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:51.560651 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.560663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.562018 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.562026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.563584 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.563598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.565291 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.565297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.565513 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.565550 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.565662 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.565857 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":25}
I20260925 09:19:51.567065 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.559579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:53893 (local address 127.3.82.2:42399)
0925 09:19:51.559779 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.559785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.560688 (+ 903us) server_negotiation.cc:407] Connection header received
0925 09:19:51.560830 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.560837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.560913 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.561012 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:51.561635 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.562859 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.564732 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.565870 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.566000 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.566122 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.566663 (+ 541us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.566790 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:19:51.567080 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.559669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:35931 (local address 127.3.82.1:39727)
0925 09:19:51.559865 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.559869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.559885 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:19:51.559950 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.559953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.560079 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.560189 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:51.560820 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.561873 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.563768 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.565102 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.565250 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.565369 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.566644 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.566806 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:19:51.567883 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.559248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42399 (local address 127.3.82.3:53893)
0925 09:19:51.560558 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.560579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.560701 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.561054 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.561059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.561104 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:51.561445 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.561455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.563024 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.563029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.564557 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.564568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.566288 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.566294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.566477 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.566509 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.567331 (+ 822us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.567487 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"mutex_wait_us":533,"thread_start_us":401,"threads_started":1}
I20260925 09:19:51.568483 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e198c2e639403d90b2f14dd64775c4" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42" is_pre_election: true
I20260925 09:19:51.568909 5454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e198c2e639403d90b2f14dd64775c4" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8358935773044bb49c0de5c30adbde61" is_pre_election: true
I20260925 09:19:51.569090 5315 raft_consensus.cc:2454] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 0.
I20260925 09:19:51.569407 5454 raft_consensus.cc:2454] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 0.
I20260925 09:19:51.569973 5527 leader_election.cc:302] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c25bdf951d74e0f81505e97284f9871, 661ee9937945416ab5760db988166a42; no voters:
I20260925 09:19:51.570442 5670 raft_consensus.cc:2781] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:51.570667 5670 raft_consensus.cc:482] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:51.570776 5670 raft_consensus.cc:3037] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:51.573882 5670 raft_consensus.cc:504] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.574558 5670 leader_election.cc:288] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 election: Requested vote from peers 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399), 661ee9937945416ab5760db988166a42 (127.3.82.1:39727)
I20260925 09:19:51.575284 5454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e198c2e639403d90b2f14dd64775c4" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8358935773044bb49c0de5c30adbde61"
I20260925 09:19:51.575500 5454 raft_consensus.cc:3037] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:51.575369 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e198c2e639403d90b2f14dd64775c4" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42"
I20260925 09:19:51.575847 5315 raft_consensus.cc:3037] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:51.578060 5454 raft_consensus.cc:2454] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 1.
I20260925 09:19:51.578469 5315 raft_consensus.cc:2454] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 1.
I20260925 09:19:51.578631 5525 leader_election.cc:302] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c25bdf951d74e0f81505e97284f9871, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:51.578971 5670 raft_consensus.cc:2781] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:51.579438 5670 raft_consensus.cc:686] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 1 LEADER]: Becoming Leader. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Running, Role: LEADER
I20260925 09:19:51.580029 5670 consensus_queue.cc:237] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.584702 5156 catalog_manager.cc:5495] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } health_report { overall_health: HEALTHY } } }
W20260925 09:19:51.617684 5644 tablet.cc:2369] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 09:19:51 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327989, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:51.664847 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.648558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36081 (local address 127.0.0.1:49038)
0925 09:19:51.649623 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.649642 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.649798 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.650308 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.650324 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.651276 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:51.651548 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.651593 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.652762 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.652766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.653798 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.653811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.654701 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.654706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.654819 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.655668 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:51.655696 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:51.658617 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:51.661887 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:51.661902 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:51.661909 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:51.662395 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:51.662940 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:51.662953 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:51.662956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:51.663187 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:51.663847 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:51.663855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:51.664022 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.664181 (+ 159us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.664421 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"thread_start_us":454,"threads_started":1}
I20260925 09:19:51.664883 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.648573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.3.82.62:36081)
0925 09:19:51.648817 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.648826 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.649791 (+ 965us) server_negotiation.cc:407] Connection header received
0925 09:19:51.649986 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.649990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.650142 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.650231 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:51.651760 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.652602 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.654046 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.654497 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.654610 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.658847 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:51.658853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:51.658856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:51.658889 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:51.661667 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.662644 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.662649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.662651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.662807 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:51.663327 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:51.663331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:51.663333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:51.663694 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:51.663825 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.664343 (+ 518us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.664572 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260925 09:19:51.671202 5156 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:49038:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 09:19:51.672700 5156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:51.681308 5434 tablet_service.cc:1467] Processing CreateTablet for tablet 98bad04307064b81a35aa5f1035d3510 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a56473f3203640c0ac407730009fc974]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:19:51.681128 5295 tablet_service.cc:1467] Processing CreateTablet for tablet 98bad04307064b81a35aa5f1035d3510 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a56473f3203640c0ac407730009fc974]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:19:51.681562 5577 tablet_service.cc:1467] Processing CreateTablet for tablet 98bad04307064b81a35aa5f1035d3510 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a56473f3203640c0ac407730009fc974]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:19:51.682175 5434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98bad04307064b81a35aa5f1035d3510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.682235 5295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98bad04307064b81a35aa5f1035d3510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.682475 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98bad04307064b81a35aa5f1035d3510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.687875 5662 tablet_bootstrap.cc:492] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: Bootstrap starting.
I20260925 09:19:51.689654 5662 tablet_bootstrap.cc:654] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.690862 5663 tablet_bootstrap.cc:492] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: Bootstrap starting.
I20260925 09:19:51.691097 5664 tablet_bootstrap.cc:492] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: Bootstrap starting.
I20260925 09:19:51.692582 5663 tablet_bootstrap.cc:654] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.693006 5664 tablet_bootstrap.cc:654] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.693341 5662 tablet_bootstrap.cc:492] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: No bootstrap required, opened a new log
I20260925 09:19:51.693497 5662 ts_tablet_manager.cc:1397] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:19:51.694593 5662 raft_consensus.cc:348] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.694965 5662 raft_consensus.cc:374] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.695079 5662 raft_consensus.cc:729] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.695477 5662 consensus_queue.cc:260] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.696002 5663 tablet_bootstrap.cc:492] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: No bootstrap required, opened a new log
I20260925 09:19:51.696154 5663 ts_tablet_manager.cc:1397] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260925 09:19:51.695992 5668 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.696293 5664 tablet_bootstrap.cc:492] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: No bootstrap required, opened a new log
I20260925 09:19:51.696400 5664 ts_tablet_manager.cc:1397] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:19:51.696393 5668 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:51.696954 5664 raft_consensus.cc:348] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.697165 5664 raft_consensus.cc:374] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.697019 5663 raft_consensus.cc:348] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.697254 5664 raft_consensus.cc:729] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.697283 5663 raft_consensus.cc:374] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.697381 5663 raft_consensus.cc:729] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee9937945416ab5760db988166a42, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.697477 5664 consensus_queue.cc:260] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.697700 5670 raft_consensus.cc:482] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:51.697633 5663 consensus_queue.cc:260] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.697873 5670 raft_consensus.cc:504] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.698415 5663 ts_tablet_manager.cc:1428] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 09:19:51.698199 5669 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.698583 5670 leader_election.cc:288] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399)
I20260925 09:19:51.698627 5669 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:51.699273 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98bad04307064b81a35aa5f1035d3510" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42" is_pre_election: true
I20260925 09:19:51.699551 5315 raft_consensus.cc:2454] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 0.
I20260925 09:19:51.699635 5454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98bad04307064b81a35aa5f1035d3510" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8358935773044bb49c0de5c30adbde61" is_pre_election: true
I20260925 09:19:51.699892 5454 raft_consensus.cc:2454] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 0.
I20260925 09:19:51.700006 5662 ts_tablet_manager.cc:1428] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260925 09:19:51.700125 5527 leader_election.cc:302] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c25bdf951d74e0f81505e97284f9871, 661ee9937945416ab5760db988166a42; no voters:
I20260925 09:19:51.700026 5670 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.700340 5670 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:51.700448 5672 raft_consensus.cc:2781] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:51.700592 5672 raft_consensus.cc:482] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:51.700747 5672 raft_consensus.cc:3037] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:51.701139 5664 ts_tablet_manager.cc:1428] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:19:51.704016 5672 raft_consensus.cc:504] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.704931 5672 leader_election.cc:288] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 election: Requested vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399)
I20260925 09:19:51.705425 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98bad04307064b81a35aa5f1035d3510" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42"
I20260925 09:19:51.705489 5454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98bad04307064b81a35aa5f1035d3510" candidate_uuid: "4c25bdf951d74e0f81505e97284f9871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8358935773044bb49c0de5c30adbde61"
I20260925 09:19:51.705704 5315 raft_consensus.cc:3037] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:51.705786 5454 raft_consensus.cc:3037] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:51.708503 5454 raft_consensus.cc:2454] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 1.
I20260925 09:19:51.708592 5315 raft_consensus.cc:2454] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c25bdf951d74e0f81505e97284f9871 in term 1.
I20260925 09:19:51.709005 5525 leader_election.cc:302] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c25bdf951d74e0f81505e97284f9871, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:51.709326 5672 raft_consensus.cc:2781] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:51.709545 5672 raft_consensus.cc:686] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [term 1 LEADER]: Becoming Leader. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Running, Role: LEADER
I20260925 09:19:51.709802 5672 consensus_queue.cc:237] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.711678 5670 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: TxnStatusTablet state changed. Reason: New leader 4c25bdf951d74e0f81505e97284f9871. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.711973 5670 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:19:51.713025 5157 catalog_manager.cc:5495] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } health_report { overall_health: HEALTHY } } }
I20260925 09:19:51.714303 5684 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 09:19:51.714597 5684 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 09:19:51.741313 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.734286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.3.82.3:38347)
0925 09:19:51.734493 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.734497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.734517 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:19:51.734868 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.734873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.734971 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.735184 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:51.735670 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.736466 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.738684 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.739171 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.739278 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.739969 (+ 691us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:51.740640 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:51.740765 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.740973 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.741062 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 09:19:51.741456 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.734100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38347 (local address 127.0.0.1:45296)
0925 09:19:51.734379 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.734406 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.734580 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.735213 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.735219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.735234 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:51.735470 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.735483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.736622 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.736629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.738470 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.738488 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.739362 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.739370 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.739627 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.739718 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:51.740778 (+ 1060us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:51.740807 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.740915 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.741099 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 09:19:51.750007 5454 raft_consensus.cc:1261] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Refusing update from remote peer 4c25bdf951d74e0f81505e97284f9871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:51.750029 5315 raft_consensus.cc:1261] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Refusing update from remote peer 4c25bdf951d74e0f81505e97284f9871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:51.751134 5670 consensus_queue.cc:1035] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:51.751515 5672 consensus_queue.cc:1035] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:51.764084 5668 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: TxnStatusTablet state changed. Reason: New leader 4c25bdf951d74e0f81505e97284f9871. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.764523 5668 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:51.768486 5670 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.768821 5670 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:19:51.770177 5672 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.770349 5669 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: TxnStatusTablet state changed. Reason: New leader 4c25bdf951d74e0f81505e97284f9871. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.770599 5672 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 4c25bdf951d74e0f81505e97284f9871: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:19:51.770661 5669 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 09:19:51.772959 5366 tablet.cc:2369] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:19:51.784466 5668 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.784770 5668 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 8358935773044bb49c0de5c30adbde61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:51.784929 5669 tablet_replica.cc:439] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c25bdf951d74e0f81505e97284f9871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } } }
I20260925 09:19:51.785218 5669 tablet_replica.cc:442] T 98bad04307064b81a35aa5f1035d3510 P 661ee9937945416ab5760db988166a42: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:19:51.795274 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.788584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.3.82.3:38347)
0925 09:19:51.788751 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.788756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.788779 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:19:51.789103 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.789108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.789220 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.789314 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:51.789822 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.790754 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.793109 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.793564 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.793703 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.794297 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:51.794417 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:51.794508 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.794765 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.794840 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:19:51.816045 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.809522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:47271 (local address 127.3.82.3:38347)
0925 09:19:51.809705 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:51.809711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:51.809732 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:19:51.809799 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:51.809805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:51.809927 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:51.810033 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:51.810553 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.811761 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.813863 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.814519 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.814653 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.815194 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:51.815388 (+ 194us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:51.815472 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:51.815643 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 09:19:51.815729 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":64}
I20260925 09:19:51.816401 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:51.809246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38347 (local address 127.3.82.3:47271)
0925 09:19:51.809477 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:51.809495 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:51.809631 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:51.810055 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:51.810059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:51.810072 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:51.810364 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.810375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.811903 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.811909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.813655 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:51.813672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:51.814692 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:51.814699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:51.814878 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:51.814964 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:51.815496 (+ 532us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:51.815514 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 09:19:51.815613 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 09:19:51.815792 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":5}
I20260925 09:19:51.828191 5315 raft_consensus.cc:1261] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Refusing update from remote peer 4c25bdf951d74e0f81505e97284f9871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:51.828280 5454 raft_consensus.cc:1261] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Refusing update from remote peer 4c25bdf951d74e0f81505e97284f9871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:51.828933 5693 consensus_queue.cc:1035] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:51.829347 5672 consensus_queue.cc:1035] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:51.835330 5687 mvcc.cc:205] Tried to move back new op lower bound from 7333183454522695680 to 7333183453516156928. Current Snapshot: MvccSnapshot[applied={T|T < 7333183454522695680}]
I20260925 09:19:51.908026 5157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:51.908345 5157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:51.912303 5157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Sending DeleteTablet for 3 replicas of tablet 45e198c2e639403d90b2f14dd64775c4
I20260925 09:19:51.913725 5434 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e198c2e639403d90b2f14dd64775c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:51 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53298
I20260925 09:19:51.913987 5295 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e198c2e639403d90b2f14dd64775c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:51 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:43784
I20260925 09:19:51.914834 5577 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e198c2e639403d90b2f14dd64775c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:51 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:45284
I20260925 09:19:51.915575 5157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:19:51.916704 5157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:51.919099 5703 tablet_replica.cc:330] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: stopping tablet replica
I20260925 09:19:51.919811 5703 raft_consensus.cc:2229] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:51.920356 5703 raft_consensus.cc:2258] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:51.923142 5703 ts_tablet_manager.cc:1905] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:51.927254 5573 tablet_service.cc:1467] Processing CreateTablet for tablet 61636a1aed3f4b72abee519d24b72ab5 (DEFAULT_TABLE table=test-table [id=8a4e948c089542efb5f9ac3e24dc2c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:51.927857 5573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61636a1aed3f4b72abee519d24b72ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.930728 5294 tablet_service.cc:1467] Processing CreateTablet for tablet 61636a1aed3f4b72abee519d24b72ab5 (DEFAULT_TABLE table=test-table [id=8a4e948c089542efb5f9ac3e24dc2c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:51.931354 5294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61636a1aed3f4b72abee519d24b72ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.931368 5705 tablet_replica.cc:330] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: stopping tablet replica
I20260925 09:19:51.932363 5705 raft_consensus.cc:2229] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:19:51.933135 5705 raft_consensus.cc:2258] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:51.934396 5433 tablet_service.cc:1467] Processing CreateTablet for tablet 61636a1aed3f4b72abee519d24b72ab5 (DEFAULT_TABLE table=test-table [id=8a4e948c089542efb5f9ac3e24dc2c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:51.934935 5433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61636a1aed3f4b72abee519d24b72ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:51.935328 5704 tablet_replica.cc:330] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: stopping tablet replica
I20260925 09:19:51.936088 5704 raft_consensus.cc:2229] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:51.937309 5664 tablet_bootstrap.cc:492] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Bootstrap starting.
I20260925 09:19:51.938723 5704 raft_consensus.cc:2258] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:51.939178 5664 tablet_bootstrap.cc:654] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.942940 5703 ts_tablet_manager.cc:1918] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:51.943222 5703 log.cc:1196] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/45e198c2e639403d90b2f14dd64775c4
I20260925 09:19:51.943907 5703 ts_tablet_manager.cc:1939] T 45e198c2e639403d90b2f14dd64775c4 P 8358935773044bb49c0de5c30adbde61: Deleting consensus metadata
I20260925 09:19:51.947419 5705 ts_tablet_manager.cc:1905] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:51.948576 5143 catalog_manager.cc:4841] TS 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399): tablet 45e198c2e639403d90b2f14dd64775c4 (table test-table [id=57b00fc546fa45b2b35ba8751f820348]) successfully deleted
I20260925 09:19:51.956562 5705 ts_tablet_manager.cc:1918] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:51.956882 5705 log.cc:1196] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/45e198c2e639403d90b2f14dd64775c4
I20260925 09:19:51.957222 5704 ts_tablet_manager.cc:1905] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:51.957618 5705 ts_tablet_manager.cc:1939] T 45e198c2e639403d90b2f14dd64775c4 P 4c25bdf951d74e0f81505e97284f9871: Deleting consensus metadata
I20260925 09:19:51.958180 5664 tablet_bootstrap.cc:492] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: No bootstrap required, opened a new log
I20260925 09:19:51.958387 5664 ts_tablet_manager.cc:1397] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.002s
I20260925 09:19:51.960203 5145 catalog_manager.cc:4841] TS 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347): tablet 45e198c2e639403d90b2f14dd64775c4 (table test-table [id=57b00fc546fa45b2b35ba8751f820348]) successfully deleted
I20260925 09:19:51.960959 5664 raft_consensus.cc:348] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.961360 5664 raft_consensus.cc:374] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.961499 5664 raft_consensus.cc:729] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.961733 5662 tablet_bootstrap.cc:492] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Bootstrap starting.
I20260925 09:19:51.961993 5664 consensus_queue.cc:260] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.962810 5664 ts_tablet_manager.cc:1428] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 09:19:51.963610 5662 tablet_bootstrap.cc:654] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.967540 5704 ts_tablet_manager.cc:1918] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:51.967742 5704 log.cc:1196] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/45e198c2e639403d90b2f14dd64775c4
I20260925 09:19:51.968569 5704 ts_tablet_manager.cc:1939] T 45e198c2e639403d90b2f14dd64775c4 P 661ee9937945416ab5760db988166a42: Deleting consensus metadata
I20260925 09:19:51.968924 5663 tablet_bootstrap.cc:492] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Bootstrap starting.
I20260925 09:19:51.970777 5663 tablet_bootstrap.cc:654] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:51.970914 5145 catalog_manager.cc:4841] TS 661ee9937945416ab5760db988166a42 (127.3.82.1:39727): tablet 45e198c2e639403d90b2f14dd64775c4 (table test-table [id=57b00fc546fa45b2b35ba8751f820348]) successfully deleted
I20260925 09:19:51.972659 5662 tablet_bootstrap.cc:492] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: No bootstrap required, opened a new log
I20260925 09:19:51.972894 5662 ts_tablet_manager.cc:1397] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260925 09:19:51.973354 5663 tablet_bootstrap.cc:492] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: No bootstrap required, opened a new log
I20260925 09:19:51.973542 5663 ts_tablet_manager.cc:1397] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260925 09:19:51.973667 5662 raft_consensus.cc:348] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.973948 5662 raft_consensus.cc:374] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.974036 5662 raft_consensus.cc:729] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.974304 5662 consensus_queue.cc:260] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.974496 5663 raft_consensus.cc:348] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.974789 5663 raft_consensus.cc:374] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:51.974867 5663 raft_consensus.cc:729] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee9937945416ab5760db988166a42, State: Initialized, Role: FOLLOWER
I20260925 09:19:51.974898 5662 ts_tablet_manager.cc:1428] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:19:51.975152 5663 consensus_queue.cc:260] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:51.975881 5663 ts_tablet_manager.cc:1428] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 09:19:52.198432 5668 raft_consensus.cc:482] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:52.198675 5668 raft_consensus.cc:504] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:52.200316 5668 leader_election.cc:288] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:52.209115 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.202014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:50327 (local address 127.3.82.3:38347)
0925 09:19:52.202214 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:52.202220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:52.202597 (+ 377us) server_negotiation.cc:407] Connection header received
0925 09:19:52.202847 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:52.202853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:52.202931 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:52.203039 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:52.203694 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.204555 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.206488 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.207896 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.208036 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.208159 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:52.208668 (+ 509us) server_negotiation.cc:299] Negotiation successful
0925 09:19:52.208798 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 09:19:52.209559 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.200753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:47091 (local address 127.3.82.1:39727)
0925 09:19:52.201326 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:52.201333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:52.201596 (+ 263us) server_negotiation.cc:407] Connection header received
0925 09:19:52.201864 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:52.201870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:52.201953 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:52.202076 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:19:52.202895 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.203823 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.206749 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.208061 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.208182 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.208277 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:52.209034 (+ 757us) server_negotiation.cc:299] Negotiation successful
0925 09:19:52.209206 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":278,"threads_started":1}
I20260925 09:19:52.209805 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.201674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38347 (local address 127.3.82.2:50327)
0925 09:19:52.202479 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:52.202495 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:52.202680 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:52.203083 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:52.203088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:52.203118 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:52.203490 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.203503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.204704 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.204711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.206331 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.206345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.208255 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.208267 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.208486 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.208511 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:19:52.208638 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 09:19:52.208776 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":1,"thread_start_us":540,"threads_started":1}
I20260925 09:19:52.209877 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.200545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39727 (local address 127.3.82.2:47091)
0925 09:19:52.201467 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:52.201489 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:52.201671 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:52.202314 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:52.202319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:52.202358 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:19:52.202655 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.202670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.203967 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.203973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.206552 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.206569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.208370 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.208379 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.208615 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.208656 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 09:19:52.208760 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 09:19:52.208919 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":1,"thread_start_us":720,"threads_started":1}
I20260925 09:19:52.210824 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61636a1aed3f4b72abee519d24b72ab5" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42" is_pre_election: true
I20260925 09:19:52.210918 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61636a1aed3f4b72abee519d24b72ab5" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c25bdf951d74e0f81505e97284f9871" is_pre_election: true
I20260925 09:19:52.211122 5315 raft_consensus.cc:2454] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 0.
I20260925 09:19:52.211421 5598 raft_consensus.cc:2454] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 0.
I20260925 09:19:52.211758 5388 leader_election.cc:302] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661ee9937945416ab5760db988166a42, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:52.212230 5668 raft_consensus.cc:2781] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:52.212375 5668 raft_consensus.cc:482] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:52.212495 5668 raft_consensus.cc:3037] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:52.214855 5668 raft_consensus.cc:504] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:52.215440 5668 leader_election.cc:288] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 election: Requested vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:52.216200 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61636a1aed3f4b72abee519d24b72ab5" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c25bdf951d74e0f81505e97284f9871"
I20260925 09:19:52.216192 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61636a1aed3f4b72abee519d24b72ab5" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42"
I20260925 09:19:52.216427 5598 raft_consensus.cc:3037] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:52.216426 5315 raft_consensus.cc:3037] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:52.219233 5598 raft_consensus.cc:2454] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 1.
I20260925 09:19:52.219741 5388 leader_election.cc:302] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c25bdf951d74e0f81505e97284f9871, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:52.219869 5315 raft_consensus.cc:2454] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 1.
I20260925 09:19:52.220077 5668 raft_consensus.cc:2781] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:52.226259 5668 raft_consensus.cc:686] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 1 LEADER]: Becoming Leader. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Running, Role: LEADER
I20260925 09:19:52.226859 5668 consensus_queue.cc:237] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:52.230858 5156 catalog_manager.cc:5495] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8358935773044bb49c0de5c30adbde61 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "8358935773044bb49c0de5c30adbde61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } health_report { overall_health: UNKNOWN } } }
I20260925 09:19:52.260298 5714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.252096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.3.82.2:42399)
0925 09:19:52.254397 (+ 2301us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:52.254402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:52.254423 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:19:52.254566 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:52.254570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:52.254701 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:52.254858 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:52.255484 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.256430 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.257818 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.258282 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.258428 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.258958 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:52.259578 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:52.259758 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:52.259844 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 09:19:52.259933 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2180,"thread_start_us":2031,"threads_started":1}
I20260925 09:19:52.282006 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.273970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:42839 (local address 127.3.82.3:38347)
0925 09:19:52.274212 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:52.274219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:52.275044 (+ 825us) server_negotiation.cc:407] Connection header received
0925 09:19:52.275273 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:52.275280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:52.275378 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:52.275502 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:52.276061 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.277436 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.279586 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.280352 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.280587 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.281117 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:52.281277 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:52.281384 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:52.281541 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 09:19:52.281636 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:19:52.282078 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.274191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38347 (local address 127.3.82.2:42839)
0925 09:19:52.274899 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:52.274921 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:52.275101 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:52.275528 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:52.275533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:52.275552 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:52.275875 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.275885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.277614 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.277620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.279407 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.279424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.280625 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.280632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.280819 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.280900 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:52.281392 (+ 492us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:52.281403 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:19:52.281482 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 09:19:52.281644 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":310,"threads_started":1}
I20260925 09:19:52.293239 5315 raft_consensus.cc:1261] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Refusing update from remote peer 8358935773044bb49c0de5c30adbde61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:52.293627 5598 raft_consensus.cc:1261] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Refusing update from remote peer 8358935773044bb49c0de5c30adbde61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:52.294400 5712 consensus_queue.cc:1035] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:52.294842 5668 consensus_queue.cc:1035] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:52.308308 5688 mvcc.cc:205] Tried to move back new op lower bound from 7333183456425750528 to 7333183456166023168. Current Snapshot: MvccSnapshot[applied={T|T < 7333183456425750528}]
I20260925 09:19:52.341205 5714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.332673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:48477 (local address 127.3.82.2:42399)
0925 09:19:52.332871 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:52.332876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:52.334375 (+ 1499us) server_negotiation.cc:407] Connection header received
0925 09:19:52.334605 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:52.334610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:52.334698 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:52.334811 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:52.335412 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.336691 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.339081 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.339576 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.339750 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.340289 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:52.340456 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:52.340575 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:52.340756 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 09:19:52.340855 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:19:52.341573 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:52.332400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42399 (local address 127.3.82.3:48477)
0925 09:19:52.334256 (+ 1856us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:52.334274 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:52.334453 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:52.334843 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:52.334848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:52.334864 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:19:52.335217 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.335231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.336885 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.336892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.338895 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:52.338914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:52.339792 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:52.339802 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:52.340008 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:52.340079 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:19:52.340590 (+ 511us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:19:52.340604 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 09:19:52.340712 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:19:52.341106 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1726,"thread_start_us":1610,"threads_started":1}
W20260925 09:19:52.343014 5688 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 09:19:53.980741 5157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:53.980983 5157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:53.986024 5157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Sending DeleteTablet for 3 replicas of tablet 61636a1aed3f4b72abee519d24b72ab5
I20260925 09:19:53.987157 5433 tablet_service.cc:1514] Processing DeleteTablet for tablet 61636a1aed3f4b72abee519d24b72ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:53 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53298
I20260925 09:19:53.987200 5577 tablet_service.cc:1514] Processing DeleteTablet for tablet 61636a1aed3f4b72abee519d24b72ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:53 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:45284
I20260925 09:19:53.987224 5294 tablet_service.cc:1514] Processing DeleteTablet for tablet 61636a1aed3f4b72abee519d24b72ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:53 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:43784
I20260925 09:19:53.998193 5749 tablet_replica.cc:330] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: stopping tablet replica
I20260925 09:19:53.998385 5748 tablet_replica.cc:330] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: stopping tablet replica
I20260925 09:19:53.998668 5749 raft_consensus.cc:2229] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:53.998692 5748 raft_consensus.cc:2229] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:19:53.999109 5749 raft_consensus.cc:2258] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:53.999392 5748 raft_consensus.cc:2258] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:54.000093 5747 tablet_replica.cc:330] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: stopping tablet replica
I20260925 09:19:54.000325 5747 raft_consensus.cc:2229] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:54.000684 5747 raft_consensus.cc:2258] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:54.001099 5748 ts_tablet_manager.cc:1905] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:54.001092 5749 ts_tablet_manager.cc:1905] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:54.001889 5747 ts_tablet_manager.cc:1905] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:54.008747 5749 ts_tablet_manager.cc:1918] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:19:54.008747 5748 ts_tablet_manager.cc:1918] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:19:54.008913 5748 log.cc:1196] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/61636a1aed3f4b72abee519d24b72ab5
I20260925 09:19:54.009025 5749 log.cc:1196] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/61636a1aed3f4b72abee519d24b72ab5
I20260925 09:19:54.009577 5749 ts_tablet_manager.cc:1939] T 61636a1aed3f4b72abee519d24b72ab5 P 661ee9937945416ab5760db988166a42: Deleting consensus metadata
I20260925 09:19:54.009569 5748 ts_tablet_manager.cc:1939] T 61636a1aed3f4b72abee519d24b72ab5 P 8358935773044bb49c0de5c30adbde61: Deleting consensus metadata
I20260925 09:19:54.010094 5747 ts_tablet_manager.cc:1918] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:19:54.010294 5747 log.cc:1196] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/61636a1aed3f4b72abee519d24b72ab5
I20260925 09:19:54.010859 5747 ts_tablet_manager.cc:1939] T 61636a1aed3f4b72abee519d24b72ab5 P 4c25bdf951d74e0f81505e97284f9871: Deleting consensus metadata
I20260925 09:19:54.010996 5145 catalog_manager.cc:4841] TS 661ee9937945416ab5760db988166a42 (127.3.82.1:39727): tablet 61636a1aed3f4b72abee519d24b72ab5 (table test-table [id=8a4e948c089542efb5f9ac3e24dc2c38]) successfully deleted
I20260925 09:19:54.011348 5143 catalog_manager.cc:4841] TS 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399): tablet 61636a1aed3f4b72abee519d24b72ab5 (table test-table [id=8a4e948c089542efb5f9ac3e24dc2c38]) successfully deleted
I20260925 09:19:54.012311 5145 catalog_manager.cc:4841] TS 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347): tablet 61636a1aed3f4b72abee519d24b72ab5 (table test-table [id=8a4e948c089542efb5f9ac3e24dc2c38]) successfully deleted
Sep 25 09:19:54 dist-test-slave-4wm5 krb5kdc[5113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:19:54 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:54.051784 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:54.035571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.3.82.62:36081)
0925 09:19:54.036247 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:54.036251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:54.036278 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:19:54.036334 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:54.036338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:54.036400 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:54.036502 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:54.037983 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:54.038870 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:54.040235 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:54.040700 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:54.040817 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:54.045466 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:54.045472 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:54.045476 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:54.045518 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:54.048668 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:54.049605 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:54.049609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:54.049612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:54.049755 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:54.050189 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:54.050193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:54.050195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:54.050619 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:54.050856 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:54.051286 (+ 430us) server_negotiation.cc:299] Negotiation successful
0925 09:19:54.051469 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":303,"threads_started":1}
I20260925 09:19:54.055769 5157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:19:54.056444 5157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:54.064226 5294 tablet_service.cc:1467] Processing CreateTablet for tablet 992efaba1a2049b09f7c0667bf555ee3 (DEFAULT_TABLE table=test-table [id=55bc1e8664ae4fc0ad4e023bc38ae48c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:54.064561 5433 tablet_service.cc:1467] Processing CreateTablet for tablet 992efaba1a2049b09f7c0667bf555ee3 (DEFAULT_TABLE table=test-table [id=55bc1e8664ae4fc0ad4e023bc38ae48c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:54.064777 5294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 992efaba1a2049b09f7c0667bf555ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:54.064733 5577 tablet_service.cc:1467] Processing CreateTablet for tablet 992efaba1a2049b09f7c0667bf555ee3 (DEFAULT_TABLE table=test-table [id=55bc1e8664ae4fc0ad4e023bc38ae48c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:54.064982 5433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 992efaba1a2049b09f7c0667bf555ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:54.065073 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 992efaba1a2049b09f7c0667bf555ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:54.070943 5758 tablet_bootstrap.cc:492] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Bootstrap starting.
I20260925 09:19:54.071971 5760 tablet_bootstrap.cc:492] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Bootstrap starting.
I20260925 09:19:54.073745 5760 tablet_bootstrap.cc:654] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:54.073753 5758 tablet_bootstrap.cc:654] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:54.075842 5759 tablet_bootstrap.cc:492] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Bootstrap starting.
I20260925 09:19:54.076257 5760 tablet_bootstrap.cc:492] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: No bootstrap required, opened a new log
I20260925 09:19:54.076265 5758 tablet_bootstrap.cc:492] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: No bootstrap required, opened a new log
I20260925 09:19:54.076398 5758 ts_tablet_manager.cc:1397] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:19:54.076398 5760 ts_tablet_manager.cc:1397] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 09:19:54.077242 5760 raft_consensus.cc:348] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.077250 5758 raft_consensus.cc:348] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.077484 5760 raft_consensus.cc:374] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:54.077504 5758 raft_consensus.cc:374] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:54.077576 5760 raft_consensus.cc:729] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Initialized, Role: FOLLOWER
I20260925 09:19:54.077605 5759 tablet_bootstrap.cc:654] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:54.077592 5758 raft_consensus.cc:729] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Initialized, Role: FOLLOWER
I20260925 09:19:54.077862 5758 consensus_queue.cc:260] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.077906 5760 consensus_queue.cc:260] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.078441 5758 ts_tablet_manager.cc:1428] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 09:19:54.078441 5760 ts_tablet_manager.cc:1428] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:19:54.082311 5759 tablet_bootstrap.cc:492] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: No bootstrap required, opened a new log
I20260925 09:19:54.082486 5759 ts_tablet_manager.cc:1397] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260925 09:19:54.083158 5759 raft_consensus.cc:348] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.083446 5759 raft_consensus.cc:374] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:54.083559 5759 raft_consensus.cc:729] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee9937945416ab5760db988166a42, State: Initialized, Role: FOLLOWER
I20260925 09:19:54.083827 5759 consensus_queue.cc:260] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.084669 5759 ts_tablet_manager.cc:1428] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:19:54.278812 5740 raft_consensus.cc:482] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:54.279033 5740 raft_consensus.cc:504] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.279775 5740 leader_election.cc:288] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:54.280509 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992efaba1a2049b09f7c0667bf555ee3" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42" is_pre_election: true
I20260925 09:19:54.280627 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992efaba1a2049b09f7c0667bf555ee3" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c25bdf951d74e0f81505e97284f9871" is_pre_election: true
I20260925 09:19:54.280819 5315 raft_consensus.cc:2454] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 0.
I20260925 09:19:54.280885 5598 raft_consensus.cc:2454] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 0.
I20260925 09:19:54.281410 5388 leader_election.cc:302] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661ee9937945416ab5760db988166a42, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:54.281755 5740 raft_consensus.cc:2781] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:54.281932 5740 raft_consensus.cc:482] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:54.282058 5740 raft_consensus.cc:3037] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:54.284344 5740 raft_consensus.cc:504] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.284962 5740 leader_election.cc:288] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 election: Requested vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:54.285579 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992efaba1a2049b09f7c0667bf555ee3" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c25bdf951d74e0f81505e97284f9871"
I20260925 09:19:54.285625 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992efaba1a2049b09f7c0667bf555ee3" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42"
I20260925 09:19:54.285867 5598 raft_consensus.cc:3037] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:54.285902 5315 raft_consensus.cc:3037] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:54.288383 5315 raft_consensus.cc:2454] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 1.
I20260925 09:19:54.288406 5598 raft_consensus.cc:2454] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 1.
I20260925 09:19:54.288960 5388 leader_election.cc:302] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661ee9937945416ab5760db988166a42, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:54.289369 5740 raft_consensus.cc:2781] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:54.289572 5740 raft_consensus.cc:686] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 1 LEADER]: Becoming Leader. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Running, Role: LEADER
I20260925 09:19:54.290050 5740 consensus_queue.cc:237] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.292940 5156 catalog_manager.cc:5495] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8358935773044bb49c0de5c30adbde61 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "8358935773044bb49c0de5c30adbde61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } health_report { overall_health: UNKNOWN } } }
I20260925 09:19:54.331341 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:54.325671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.3.82.2:42399)
0925 09:19:54.326474 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:54.326479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:54.326498 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:19:54.326558 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:54.326561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:54.326628 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:54.326768 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:19:54.327290 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:54.328089 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:54.329391 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:54.329895 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:54.330030 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:54.330468 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:19:54.330650 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:19:54.330750 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:54.330897 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 09:19:54.330991 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":455,"threads_started":1}
I20260925 09:19:54.342549 5598 raft_consensus.cc:1261] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Refusing update from remote peer 8358935773044bb49c0de5c30adbde61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:54.342543 5315 raft_consensus.cc:1261] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Refusing update from remote peer 8358935773044bb49c0de5c30adbde61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:54.343353 5740 consensus_queue.cc:1035] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:54.343715 5736 consensus_queue.cc:1035] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:54.400497 5158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:54.400831 5158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:54.404008 5158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Sending DeleteTablet for 3 replicas of tablet 992efaba1a2049b09f7c0667bf555ee3
I20260925 09:19:54.405042 5294 tablet_service.cc:1514] Processing DeleteTablet for tablet 992efaba1a2049b09f7c0667bf555ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:54 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:43784
I20260925 09:19:54.405120 5433 tablet_service.cc:1514] Processing DeleteTablet for tablet 992efaba1a2049b09f7c0667bf555ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:54 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53298
I20260925 09:19:54.405367 5749 tablet_replica.cc:330] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: stopping tablet replica
I20260925 09:19:54.405395 5748 tablet_replica.cc:330] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: stopping tablet replica
I20260925 09:19:54.405334 5577 tablet_service.cc:1514] Processing DeleteTablet for tablet 992efaba1a2049b09f7c0667bf555ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:54 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:45284
I20260925 09:19:54.405562 5749 raft_consensus.cc:2229] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:54.405579 5748 raft_consensus.cc:2229] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:19:54.405803 5749 raft_consensus.cc:2258] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:54.405972 5748 raft_consensus.cc:2258] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:54.406250 5747 tablet_replica.cc:330] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: stopping tablet replica
I20260925 09:19:54.406499 5747 raft_consensus.cc:2229] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:54.406687 5747 raft_consensus.cc:2258] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:54.406714 5158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:19:54.406997 5749 ts_tablet_manager.cc:1905] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:54.407228 5748 ts_tablet_manager.cc:1905] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:19:54.407414 5158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:19:54.407804 5747 ts_tablet_manager.cc:1905] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:54.413657 5749 ts_tablet_manager.cc:1918] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:54.413767 5748 ts_tablet_manager.cc:1918] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:54.413951 5748 log.cc:1196] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/992efaba1a2049b09f7c0667bf555ee3
I20260925 09:19:54.413805 5749 log.cc:1196] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/992efaba1a2049b09f7c0667bf555ee3
I20260925 09:19:54.414363 5748 ts_tablet_manager.cc:1939] T 992efaba1a2049b09f7c0667bf555ee3 P 8358935773044bb49c0de5c30adbde61: Deleting consensus metadata
I20260925 09:19:54.414717 5749 ts_tablet_manager.cc:1939] T 992efaba1a2049b09f7c0667bf555ee3 P 661ee9937945416ab5760db988166a42: Deleting consensus metadata
I20260925 09:19:54.414731 5747 ts_tablet_manager.cc:1918] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:19:54.414873 5747 log.cc:1196] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/992efaba1a2049b09f7c0667bf555ee3
I20260925 09:19:54.415304 5747 ts_tablet_manager.cc:1939] T 992efaba1a2049b09f7c0667bf555ee3 P 4c25bdf951d74e0f81505e97284f9871: Deleting consensus metadata
I20260925 09:19:54.415650 5294 tablet_service.cc:1467] Processing CreateTablet for tablet 06c71337c0b445b38843357243b6422a (DEFAULT_TABLE table=test-table [id=dcee02c0f8e74969a77b053a8fd605d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:54.415723 5143 catalog_manager.cc:4841] TS 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399): tablet 992efaba1a2049b09f7c0667bf555ee3 (table test-table [id=55bc1e8664ae4fc0ad4e023bc38ae48c]) successfully deleted
I20260925 09:19:54.415670 5577 tablet_service.cc:1467] Processing CreateTablet for tablet 06c71337c0b445b38843357243b6422a (DEFAULT_TABLE table=test-table [id=dcee02c0f8e74969a77b053a8fd605d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:54.416138 5294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c71337c0b445b38843357243b6422a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:54.416405 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c71337c0b445b38843357243b6422a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:54.416476 5433 tablet_service.cc:1467] Processing CreateTablet for tablet 06c71337c0b445b38843357243b6422a (DEFAULT_TABLE table=test-table [id=dcee02c0f8e74969a77b053a8fd605d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:19:54.416888 5433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c71337c0b445b38843357243b6422a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:54.417068 5145 catalog_manager.cc:4841] TS 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347): tablet 992efaba1a2049b09f7c0667bf555ee3 (table test-table [id=55bc1e8664ae4fc0ad4e023bc38ae48c]) successfully deleted
I20260925 09:19:54.421563 5759 tablet_bootstrap.cc:492] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Bootstrap starting.
I20260925 09:19:54.421850 5145 catalog_manager.cc:4841] TS 661ee9937945416ab5760db988166a42 (127.3.82.1:39727): tablet 992efaba1a2049b09f7c0667bf555ee3 (table test-table [id=55bc1e8664ae4fc0ad4e023bc38ae48c]) successfully deleted
I20260925 09:19:54.423300 5759 tablet_bootstrap.cc:654] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:54.423401 5760 tablet_bootstrap.cc:492] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Bootstrap starting.
I20260925 09:19:54.425450 5760 tablet_bootstrap.cc:654] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:54.426095 5758 tablet_bootstrap.cc:492] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Bootstrap starting.
I20260925 09:19:54.427699 5758 tablet_bootstrap.cc:654] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:54.429805 5758 tablet_bootstrap.cc:492] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: No bootstrap required, opened a new log
I20260925 09:19:54.430087 5758 ts_tablet_manager.cc:1397] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260925 09:19:54.430832 5758 raft_consensus.cc:348] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.431054 5758 raft_consensus.cc:374] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:54.431161 5758 raft_consensus.cc:729] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c25bdf951d74e0f81505e97284f9871, State: Initialized, Role: FOLLOWER
I20260925 09:19:54.431488 5758 consensus_queue.cc:260] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.432083 5758 ts_tablet_manager.cc:1428] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 09:19:54.434161 5760 tablet_bootstrap.cc:492] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: No bootstrap required, opened a new log
I20260925 09:19:54.434353 5760 ts_tablet_manager.cc:1397] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20260925 09:19:54.434567 5759 tablet_bootstrap.cc:492] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: No bootstrap required, opened a new log
I20260925 09:19:54.434770 5759 ts_tablet_manager.cc:1397] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260925 09:19:54.435166 5760 raft_consensus.cc:348] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.435449 5760 raft_consensus.cc:374] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:54.435541 5760 raft_consensus.cc:729] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Initialized, Role: FOLLOWER
I20260925 09:19:54.435528 5759 raft_consensus.cc:348] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.435781 5759 raft_consensus.cc:374] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:54.435865 5759 raft_consensus.cc:729] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661ee9937945416ab5760db988166a42, State: Initialized, Role: FOLLOWER
I20260925 09:19:54.435787 5760 consensus_queue.cc:260] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.436136 5759 consensus_queue.cc:260] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.436388 5760 ts_tablet_manager.cc:1428] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 09:19:54.436749 5759 ts_tablet_manager.cc:1428] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 09:19:54.446184 5740 raft_consensus.cc:482] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:19:54.446364 5740 raft_consensus.cc:504] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.447088 5740 leader_election.cc:288] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:54.447657 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c71337c0b445b38843357243b6422a" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42" is_pre_election: true
I20260925 09:19:54.447934 5315 raft_consensus.cc:2454] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 0.
I20260925 09:19:54.448191 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c71337c0b445b38843357243b6422a" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c25bdf951d74e0f81505e97284f9871" is_pre_election: true
I20260925 09:19:54.448431 5388 leader_election.cc:302] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661ee9937945416ab5760db988166a42, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:54.448475 5598 raft_consensus.cc:2454] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 0.
I20260925 09:19:54.448746 5740 raft_consensus.cc:2781] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:19:54.448863 5740 raft_consensus.cc:482] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:19:54.449012 5740 raft_consensus.cc:3037] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:54.451484 5740 raft_consensus.cc:504] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.452070 5740 leader_election.cc:288] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 election: Requested vote from peers 661ee9937945416ab5760db988166a42 (127.3.82.1:39727), 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347)
I20260925 09:19:54.452749 5315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c71337c0b445b38843357243b6422a" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661ee9937945416ab5760db988166a42"
I20260925 09:19:54.452857 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c71337c0b445b38843357243b6422a" candidate_uuid: "8358935773044bb49c0de5c30adbde61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c25bdf951d74e0f81505e97284f9871"
I20260925 09:19:54.452981 5315 raft_consensus.cc:3037] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:54.453121 5598 raft_consensus.cc:3037] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:54.455940 5598 raft_consensus.cc:2454] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 1.
I20260925 09:19:54.455955 5315 raft_consensus.cc:2454] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8358935773044bb49c0de5c30adbde61 in term 1.
I20260925 09:19:54.456499 5388 leader_election.cc:302] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c25bdf951d74e0f81505e97284f9871, 8358935773044bb49c0de5c30adbde61; no voters:
I20260925 09:19:54.456871 5740 raft_consensus.cc:2781] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:54.457232 5740 raft_consensus.cc:686] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 1 LEADER]: Becoming Leader. State: Replica: 8358935773044bb49c0de5c30adbde61, State: Running, Role: LEADER
I20260925 09:19:54.457573 5740 consensus_queue.cc:237] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } }
I20260925 09:19:54.460835 5158 catalog_manager.cc:5495] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8358935773044bb49c0de5c30adbde61 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "8358935773044bb49c0de5c30adbde61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8358935773044bb49c0de5c30adbde61" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 } health_report { overall_health: UNKNOWN } } }
I20260925 09:19:54.494738 5315 raft_consensus.cc:1261] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Refusing update from remote peer 8358935773044bb49c0de5c30adbde61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:54.494738 5598 raft_consensus.cc:1261] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Refusing update from remote peer 8358935773044bb49c0de5c30adbde61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:19:54.495338 5740 consensus_queue.cc:1035] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661ee9937945416ab5760db988166a42" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:54.495630 5724 consensus_queue.cc:1035] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c25bdf951d74e0f81505e97284f9871" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:19:54.501166 5730 mvcc.cc:205] Tried to move back new op lower bound from 7333183465444769792 to 7333183465301069824. Current Snapshot: MvccSnapshot[applied={T|T < 7333183465444769792}]
W20260925 09:19:54.516620 5729 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 09:19:56.128545 5156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:56.128829 5156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:19:56.133646 5156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4887fa3c2e104c5583bc702051bef7cb: Sending DeleteTablet for 3 replicas of tablet 06c71337c0b445b38843357243b6422a
I20260925 09:19:56.134634 5294 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c71337c0b445b38843357243b6422a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:56 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:43784
I20260925 09:19:56.134747 5433 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c71337c0b445b38843357243b6422a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:56 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53298
I20260925 09:19:56.134722 5577 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c71337c0b445b38843357243b6422a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:19:56 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:45284
I20260925 09:19:56.143232 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5229
I20260925 09:19:56.150107 5801 tablet_replica.cc:330] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: stopping tablet replica
I20260925 09:19:56.150210 5802 tablet_replica.cc:330] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: stopping tablet replica
I20260925 09:19:56.150403 5802 raft_consensus.cc:2229] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:19:56.150401 5801 raft_consensus.cc:2229] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:56.150830 5801 raft_consensus.cc:2258] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:56.151021 5803 tablet_replica.cc:330] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: stopping tablet replica
I20260925 09:19:56.151221 5803 raft_consensus.cc:2229] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:19:56.151460 5803 raft_consensus.cc:2258] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:56.151432 5802 raft_consensus.cc:2258] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:19:56.152410 5801 ts_tablet_manager.cc:1905] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:56.152944 5803 ts_tablet_manager.cc:1905] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:56.153340 5802 ts_tablet_manager.cc:1905] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:19:56.160378 5803 ts_tablet_manager.cc:1918] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:19:56.160552 5803 log.cc:1196] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/06c71337c0b445b38843357243b6422a
I20260925 09:19:56.161013 5802 ts_tablet_manager.cc:1918] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:19:56.161087 5803 ts_tablet_manager.cc:1939] T 06c71337c0b445b38843357243b6422a P 4c25bdf951d74e0f81505e97284f9871: Deleting consensus metadata
I20260925 09:19:56.161211 5802 log.cc:1196] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/06c71337c0b445b38843357243b6422a
I20260925 09:19:56.161713 5802 ts_tablet_manager.cc:1939] T 06c71337c0b445b38843357243b6422a P 8358935773044bb49c0de5c30adbde61: Deleting consensus metadata
I20260925 09:19:56.162322 5801 ts_tablet_manager.cc:1918] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 09:19:56.162609 5145 catalog_manager.cc:4841] TS 4c25bdf951d74e0f81505e97284f9871 (127.3.82.3:38347): tablet 06c71337c0b445b38843357243b6422a (table test-table [id=dcee02c0f8e74969a77b053a8fd605d0]) successfully deleted
I20260925 09:19:56.163337 5801 log.cc:1196] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/06c71337c0b445b38843357243b6422a
I20260925 09:19:56.163457 5143 catalog_manager.cc:4841] TS 8358935773044bb49c0de5c30adbde61 (127.3.82.2:42399): tablet 06c71337c0b445b38843357243b6422a (table test-table [id=dcee02c0f8e74969a77b053a8fd605d0]) successfully deleted
I20260925 09:19:56.163856 5801 ts_tablet_manager.cc:1939] T 06c71337c0b445b38843357243b6422a P 661ee9937945416ab5760db988166a42: Deleting consensus metadata
I20260925 09:19:56.165145 5145 catalog_manager.cc:4841] TS 661ee9937945416ab5760db988166a42 (127.3.82.1:39727): tablet 06c71337c0b445b38843357243b6422a (table test-table [id=dcee02c0f8e74969a77b053a8fd605d0]) successfully deleted
Sep 25 09:19:56 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:19:56.196691 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:56.144997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.3.82.1:39727)
0925 09:19:56.152057 (+ 7060us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:56.152065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:56.152828 (+ 763us) server_negotiation.cc:407] Connection header received
0925 09:19:56.152900 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:56.152904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:56.153007 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:56.153121 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:56.155147 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:56.156568 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:56.158669 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:56.159751 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:56.159990 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:56.189574 (+ 29584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:56.189604 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:56.189621 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:56.189668 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:56.193067 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:56.194098 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:56.194107 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:56.194110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:56.194298 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:56.194896 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:56.194901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:56.194904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:56.195386 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:56.195644 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:56.195958 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 09:19:56.196249 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6865,"thread_start_us":7274,"threads_started":1}
W20260925 09:19:56.197378 5360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43794
W20260925 09:19:56.198091 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5229: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:19:56.198343 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 5229
I20260925 09:19:56.217238 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5369
Sep 25 09:19:56 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:19:56.236747 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:56.218407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.3.82.2:42399)
0925 09:19:56.218979 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:56.218984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:56.219001 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:19:56.219068 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:56.219072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:56.219135 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:56.219232 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:56.220758 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:56.221801 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:56.223274 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:56.223808 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:56.223936 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:56.229247 (+ 5311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:56.229267 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:56.229285 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:56.229354 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:56.232971 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:56.233981 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:56.233996 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:56.234003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:56.234319 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:56.234803 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:56.234807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:56.234809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:56.235324 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:56.235563 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:56.235996 (+ 433us) server_negotiation.cc:299] Negotiation successful
0925 09:19:56.236338 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":341,"threads_started":1}
W20260925 09:19:56.237520 5499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53314
W20260925 09:19:56.238425 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:19:56.238608 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 5369
I20260925 09:19:56.264139 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5508
Sep 25 09:19:56 dist-test-slave-4wm5 krb5kdc[5113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:19:56.282299 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:56.265006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.3.82.3:38347)
0925 09:19:56.265591 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:56.265596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:56.265611 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:19:56.265683 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:56.265687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:56.265765 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:56.265937 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:56.267329 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:56.268123 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:56.269375 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:56.269882 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:56.270030 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:56.274942 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:56.274971 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:56.274989 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:56.275035 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:56.278713 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:56.279713 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:56.279725 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:56.279727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:56.279904 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:56.280475 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:56.280479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:56.280482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:56.280978 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:56.281274 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:56.281651 (+ 377us) server_negotiation.cc:299] Negotiation successful
0925 09:19:56.281965 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":260,"threads_started":1}
W20260925 09:19:56.283034 5638 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45302
W20260925 09:19:56.283918 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5508: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:19:56.284085 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 5508
I20260925 09:19:56.309620 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5126
I20260925 09:19:56.310945 5198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:19:56.443357 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 5126
2026-09-25T09:19:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8685 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:19:56 dist-test-slave-4wm5 krb5kdc[5811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:19:56 dist-test-slave-4wm5 krb5kdc[5811](info): set up 2 sockets
Sep 25 09:19:56 dist-test-slave-4wm5 krb5kdc[5811](info): commencing operation
krb5kdc: starting...
W20260925 09:19:57.579017 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:19:57 dist-test-slave-4wm5 krb5kdc[5811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:19:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:19:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:19:57.790829 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:44639
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:44639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:58.140177 5824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:58.140506 5824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:58.140596 5824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:58.150447 5824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:19:58.150601 5824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:58.150664 5824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:58.150750 5824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:19:58.150820 5824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:19:58.163453 5824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:44639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:44639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:58.164995 5824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:58.166572 5824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:58.180150 5832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:58.180227 5829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:58.180207 5830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:58.182044 5824 server_base.cc:1034] running on GCE node
I20260925 09:19:58.183063 5824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:58.184950 5824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:58.186228 5824 hybrid_clock.cc:648] HybridClock initialized: now 1790327998186174 us; error 45 us; skew 500 ppm
I20260925 09:19:58.186643 5824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:58 dist-test-slave-4wm5 krb5kdc[5811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327998, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:58.190434 5824 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:19:58.192158 5824 webserver.cc:460] Webserver started at http://127.3.82.62:44413/ using document root <none> and password file <none>
I20260925 09:19:58.192929 5824 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:58.193039 5824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:58.193419 5824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:58.196863 5824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "e6dd5b6e632f442caf8d3004feb60f88"
format_stamp: "Formatted at 2026-09-25 09:19:58 on dist-test-slave-4wm5"
server_key: "4c085a8bbacb7944361d54d1375999e7"
server_key_iv: "f05a297070d781db5c4a6677bfc9888a"
server_key_version: "encryptionkey@0"
I20260925 09:19:58.197739 5824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "e6dd5b6e632f442caf8d3004feb60f88"
format_stamp: "Formatted at 2026-09-25 09:19:58 on dist-test-slave-4wm5"
server_key: "4c085a8bbacb7944361d54d1375999e7"
server_key_iv: "f05a297070d781db5c4a6677bfc9888a"
server_key_version: "encryptionkey@0"
I20260925 09:19:58.203977 5824 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260925 09:19:58.208498 5839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:58.210362 5824 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:19:58.210613 5824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "e6dd5b6e632f442caf8d3004feb60f88"
format_stamp: "Formatted at 2026-09-25 09:19:58 on dist-test-slave-4wm5"
server_key: "4c085a8bbacb7944361d54d1375999e7"
server_key_iv: "f05a297070d781db5c4a6677bfc9888a"
server_key_version: "encryptionkey@0"
I20260925 09:19:58.210879 5824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:58.253794 5824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:58.259459 5824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:58.259796 5824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:58.281283 5824 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:44639
I20260925 09:19:58.281399 5890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:44639 every 8 connection(s)
I20260925 09:19:58.282898 5824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:19:58.286409 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 5824
I20260925 09:19:58.286677 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:19:58.287288 5891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:19:58.287319 3400 external_mini_cluster.cc:1428] Setting key 662270a190e1536e1c377efb1d73b3cd
I20260925 09:19:58.303128 5891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88: Bootstrap starting.
Sep 25 09:19:58 dist-test-slave-4wm5 krb5kdc[5811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:58.307319 5891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88: Neither blocks nor log segments found. Creating new log.
I20260925 09:19:58.308794 5891 log.cc:824] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88: Log is configured to *not* fsync() on all Append() calls
I20260925 09:19:58.312645 5891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88: No bootstrap required, opened a new log
I20260925 09:19:58.314952 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:58.289376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.3.82.62:44639)
0925 09:19:58.290263 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:58.290275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:58.290324 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:19:58.292560 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:58.292600 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:58.293142 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:58.293578 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:58.295527 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:58.297003 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:58.298915 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:58.299678 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:58.299833 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:58.305110 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:58.305124 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:58.305148 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:58.305202 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:58.309428 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:58.310687 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:58.310695 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:58.310704 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:58.310931 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:58.311438 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:58.311442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:58.311446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:58.312158 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:58.312545 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:58.312993 (+ 448us) server_negotiation.cc:299] Negotiation successful
0925 09:19:58.313521 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":408,"threads_started":1}
I20260925 09:19:58.322229 5891 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6dd5b6e632f442caf8d3004feb60f88" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44639 } }
I20260925 09:19:58.322631 5891 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:19:58.322715 5891 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6dd5b6e632f442caf8d3004feb60f88, State: Initialized, Role: FOLLOWER
I20260925 09:19:58.323370 5891 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6dd5b6e632f442caf8d3004feb60f88" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44639 } }
I20260925 09:19:58.323588 5891 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:19:58.323684 5891 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:19:58.323853 5891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:19:58.326134 5891 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6dd5b6e632f442caf8d3004feb60f88" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44639 } }
I20260925 09:19:58.326637 5891 leader_election.cc:302] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6dd5b6e632f442caf8d3004feb60f88; no voters:
I20260925 09:19:58.327468 5891 leader_election.cc:288] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:19:58.327659 5896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:19:58.328377 5896 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [term 1 LEADER]: Becoming Leader. State: Replica: e6dd5b6e632f442caf8d3004feb60f88, State: Running, Role: LEADER
I20260925 09:19:58.328985 5896 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6dd5b6e632f442caf8d3004feb60f88" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44639 } }
I20260925 09:19:58.330013 5891 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:19:58.332036 5897 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6dd5b6e632f442caf8d3004feb60f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6dd5b6e632f442caf8d3004feb60f88" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44639 } } }
I20260925 09:19:58.332103 5898 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6dd5b6e632f442caf8d3004feb60f88. Latest consensus state: current_term: 1 leader_uuid: "e6dd5b6e632f442caf8d3004feb60f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6dd5b6e632f442caf8d3004feb60f88" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44639 } } }
I20260925 09:19:58.332495 5898 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:58.332495 5897 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88 [sys.catalog]: This master's current role is: LEADER
I20260925 09:19:58.334723 5903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:19:58.340874 5903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:19:58.351317 5903 catalog_manager.cc:1347] Generated new cluster ID: 04e361002288496dafd1984c7d80fac8
I20260925 09:19:58.351431 5903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:19:58.383188 5903 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:19:58.384781 5903 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:19:58.394049 5903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6dd5b6e632f442caf8d3004feb60f88: Generated new TSK 0
I20260925 09:19:58.395049 5903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:19:58.450809 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:44639
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:58.804944 5917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:58.805264 5917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:58.805437 5917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:58.812935 5917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:58.813054 5917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:58.813251 5917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:19:58.823621 5917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:44639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:58.825045 5917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:58.826531 5917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:58.841275 5922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:58.841384 5923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:58.841369 5925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:58.842746 5917 server_base.cc:1034] running on GCE node
I20260925 09:19:58.843415 5917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:58.844440 5917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:58.845666 5917 hybrid_clock.cc:648] HybridClock initialized: now 1790327998845601 us; error 48 us; skew 500 ppm
I20260925 09:19:58.846052 5917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:58 dist-test-slave-4wm5 krb5kdc[5811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327998, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:58.849529 5917 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:19:58.851163 5917 webserver.cc:460] Webserver started at http://127.3.82.1:38331/ using document root <none> and password file <none>
I20260925 09:19:58.851887 5917 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:58.851992 5917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:58.852353 5917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:58.855489 5917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "be77531f0ad647dd9e6d86ce775164dc"
format_stamp: "Formatted at 2026-09-25 09:19:58 on dist-test-slave-4wm5"
server_key: "a1e80fed49e4c0054ccc66c3f15439fa"
server_key_iv: "92fc9984fe40120198fc766452389511"
server_key_version: "encryptionkey@0"
I20260925 09:19:58.856283 5917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "be77531f0ad647dd9e6d86ce775164dc"
format_stamp: "Formatted at 2026-09-25 09:19:58 on dist-test-slave-4wm5"
server_key: "a1e80fed49e4c0054ccc66c3f15439fa"
server_key_iv: "92fc9984fe40120198fc766452389511"
server_key_version: "encryptionkey@0"
I20260925 09:19:58.862023 5917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:19:58.866250 5932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:58.867887 5917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:19:58.868114 5917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "be77531f0ad647dd9e6d86ce775164dc"
format_stamp: "Formatted at 2026-09-25 09:19:58 on dist-test-slave-4wm5"
server_key: "a1e80fed49e4c0054ccc66c3f15439fa"
server_key_iv: "92fc9984fe40120198fc766452389511"
server_key_version: "encryptionkey@0"
I20260925 09:19:58.868340 5917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:58.896842 5917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:58.901803 5917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:58.902189 5917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:58.903285 5917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:58.904709 5917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:58.904784 5917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:58.904932 5917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:58.904999 5917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:58.940898 5917 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:45695
I20260925 09:19:58.940949 6044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:45695 every 8 connection(s)
I20260925 09:19:58.942353 5917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:19:58.944245 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 5917
I20260925 09:19:58.944456 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:19:58.945055 3400 external_mini_cluster.cc:1428] Setting key 8bc225c763ceea2f66e64ce9db7e13d0
Sep 25 09:19:58 dist-test-slave-4wm5 krb5kdc[5811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327998, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:19:58.969511 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:58.945396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:35813 (local address 127.3.82.62:44639)
0925 09:19:58.945926 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:58.945933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:58.947077 (+ 1144us) server_negotiation.cc:407] Connection header received
0925 09:19:58.950245 (+ 3168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:58.950250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:58.950335 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:58.950485 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:58.952894 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:58.953672 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:58.955533 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:58.956007 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:58.956119 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:58.960902 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:58.960912 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:58.960920 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:58.960974 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:58.965289 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:58.966529 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:58.966536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:58.966541 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:58.966726 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:58.967352 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:58.967358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:58.967362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:58.967780 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:58.967986 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:58.968938 (+ 952us) server_negotiation.cc:299] Negotiation successful
0925 09:19:58.969247 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":228,"threads_started":1}
I20260925 09:19:58.970562 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:58.946114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:44639 (local address 127.3.82.1:35813)
0925 09:19:58.946909 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:58.946955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:58.949698 (+ 2743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:58.950807 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:58.950822 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:58.951719 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:58.952635 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:58.952665 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:58.953945 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:58.953949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:58.955348 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:58.955366 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:58.956291 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:58.956301 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:58.956671 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:58.957515 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:58.957553 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:58.960568 (+ 3015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:58.965535 (+ 4967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:58.965544 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:58.965560 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:58.966337 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:58.966898 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:58.966903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:58.966905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:58.967141 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:58.967985 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:58.967993 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:58.968391 (+ 398us) client_negotiation.cc:769] Sending connection context
0925 09:19:58.968683 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 09:19:58.969257 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":341,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:19:58.973125 6045 heartbeater.cc:347] Connected to a master server at 127.3.82.62:44639
I20260925 09:19:58.973495 6045 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:58.974481 6045 heartbeater.cc:511] Master 127.3.82.62:44639 requested a full tablet report, sending...
I20260925 09:19:58.977136 5856 ts_manager.cc:194] Registered new tserver with Master: be77531f0ad647dd9e6d86ce775164dc (127.3.82.1:45695)
I20260925 09:19:58.979424 5856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:35813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:19:59.003075 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:44639
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:59.343523 6051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:59.343925 6051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:59.344102 6051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:59.351766 6051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:59.351949 6051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:59.352156 6051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:19:59.362196 6051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:44639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:59.363610 6051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:59.364984 6051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:59.380188 6056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:59.380218 6057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:59.380136 6059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:59.381093 6051 server_base.cc:1034] running on GCE node
I20260925 09:19:59.381865 6051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:59.383008 6051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:59.384277 6051 hybrid_clock.cc:648] HybridClock initialized: now 1790327999384221 us; error 43 us; skew 500 ppm
I20260925 09:19:59.384660 6051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:59 dist-test-slave-4wm5 krb5kdc[5811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:59.388617 6051 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:19:59.390774 6051 webserver.cc:460] Webserver started at http://127.3.82.2:45507/ using document root <none> and password file <none>
I20260925 09:19:59.391765 6051 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:59.391919 6051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:59.392292 6051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:59.395709 6051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "764c32eacf18404da114b1f38a5ef9f9"
format_stamp: "Formatted at 2026-09-25 09:19:59 on dist-test-slave-4wm5"
server_key: "8243c2de2dbd9f3784b17f3896d5ac17"
server_key_iv: "811c9441af4181db12c52eee93873a1b"
server_key_version: "encryptionkey@0"
I20260925 09:19:59.396653 6051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "764c32eacf18404da114b1f38a5ef9f9"
format_stamp: "Formatted at 2026-09-25 09:19:59 on dist-test-slave-4wm5"
server_key: "8243c2de2dbd9f3784b17f3896d5ac17"
server_key_iv: "811c9441af4181db12c52eee93873a1b"
server_key_version: "encryptionkey@0"
I20260925 09:19:59.402694 6051 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260925 09:19:59.406633 6066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:59.408277 6051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:19:59.408586 6051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "764c32eacf18404da114b1f38a5ef9f9"
format_stamp: "Formatted at 2026-09-25 09:19:59 on dist-test-slave-4wm5"
server_key: "8243c2de2dbd9f3784b17f3896d5ac17"
server_key_iv: "811c9441af4181db12c52eee93873a1b"
server_key_version: "encryptionkey@0"
I20260925 09:19:59.408823 6051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:59.427754 6051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:59.433266 6051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:59.433704 6051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:59.435048 6051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:59.436641 6051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:59.436733 6051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:59.436895 6051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:59.436947 6051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:59.473938 6051 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:36003
I20260925 09:19:59.473979 6178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:36003 every 8 connection(s)
I20260925 09:19:59.475364 6051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:19:59.484139 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6051
I20260925 09:19:59.484442 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:19:59.484901 3400 external_mini_cluster.cc:1428] Setting key a869e8f40797b51dae9b5512bcff863d
Sep 25 09:19:59 dist-test-slave-4wm5 krb5kdc[5811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:19:59.499595 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:59.478387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:45021 (local address 127.3.82.62:44639)
0925 09:19:59.478888 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0925 09:19:59.478892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:19:59.479827 (+ 935us) server_negotiation.cc:407] Connection header received
0925 09:19:59.481724 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:19:59.481729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:19:59.481871 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:19:59.482002 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:19:59.483697 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:59.484539 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:59.486533 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:19:59.487096 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:19:59.487231 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:59.492584 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:19:59.492594 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:19:59.492601 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:19:59.492663 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:19:59.495754 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:59.496929 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:59.496934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:59.496937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:59.497092 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:19:59.497634 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:19:59.497638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:19:59.497640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:19:59.498026 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:19:59.498143 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 09:19:59.499095 (+ 952us) server_negotiation.cc:299] Negotiation successful
0925 09:19:59.499314 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":289,"threads_started":1}
I20260925 09:19:59.501473 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:19:59.478850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:44639 (local address 127.3.82.2:45021)
0925 09:19:59.479646 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0925 09:19:59.479691 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:19:59.481424 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:19:59.482268 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:19:59.482277 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:19:59.482857 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:19:59.483484 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:59.483503 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:59.484725 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:59.484729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:59.486365 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:19:59.486380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:19:59.487348 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:19:59.487356 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:19:59.487753 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:19:59.488664 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:19:59.488708 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:19:59.492140 (+ 3432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:19:59.495951 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:59.495965 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:59.495988 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:59.496770 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:59.497223 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:19:59.497231 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:19:59.497237 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:19:59.497504 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:19:59.498169 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:19:59.498180 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:19:59.498683 (+ 503us) client_negotiation.cc:769] Sending connection context
0925 09:19:59.499013 (+ 330us) client_negotiation.cc:240] Negotiation successful
0925 09:19:59.499663 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":374,"threads_started":1}
I20260925 09:19:59.503397 6179 heartbeater.cc:347] Connected to a master server at 127.3.82.62:44639
I20260925 09:19:59.503777 6179 heartbeater.cc:464] Registering TS with master...
I20260925 09:19:59.504550 6179 heartbeater.cc:511] Master 127.3.82.62:44639 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:19:59.506168 5856 ts_manager.cc:194] Registered new tserver with Master: 764c32eacf18404da114b1f38a5ef9f9 (127.3.82.2:36003)
I20260925 09:19:59.507097 5856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:45021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:19:59.534430 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:44639
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:19:59.874287 6185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:19:59.874653 6185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:19:59.874831 6185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:19:59.883070 6185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:19:59.883251 6185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:19:59.883435 6185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:19:59.893787 6185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:44639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:19:59.895285 6185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:19:59.896632 6185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:19:59.910831 6190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:59.910902 6191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:19:59.910912 6193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:19:59.911695 6185 server_base.cc:1034] running on GCE node
I20260925 09:19:59.912307 6185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:19:59.913394 6185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:19:59.914605 6185 hybrid_clock.cc:648] HybridClock initialized: now 1790327999914515 us; error 88 us; skew 500 ppm
I20260925 09:19:59.915031 6185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:19:59 dist-test-slave-4wm5 krb5kdc[5811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:19:59.918970 6185 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:19:59.920765 6185 webserver.cc:460] Webserver started at http://127.3.82.3:44161/ using document root <none> and password file <none>
I20260925 09:19:59.921540 6185 fs_manager.cc:359] Metadata directory not provided
I20260925 09:19:59.921703 6185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:19:59.922106 6185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:19:59.925381 6185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "459c112bc02c4c778d5cfbc6687ed991"
format_stamp: "Formatted at 2026-09-25 09:19:59 on dist-test-slave-4wm5"
server_key: "cba721ec6695243005f17479524e51b1"
server_key_iv: "b1cb468a89aa248848d5e67be3bf0e98"
server_key_version: "encryptionkey@0"
I20260925 09:19:59.926323 6185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "459c112bc02c4c778d5cfbc6687ed991"
format_stamp: "Formatted at 2026-09-25 09:19:59 on dist-test-slave-4wm5"
server_key: "cba721ec6695243005f17479524e51b1"
server_key_iv: "b1cb468a89aa248848d5e67be3bf0e98"
server_key_version: "encryptionkey@0"
I20260925 09:19:59.931983 6185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:19:59.935989 6200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:59.937683 6185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:19:59.937983 6185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "459c112bc02c4c778d5cfbc6687ed991"
format_stamp: "Formatted at 2026-09-25 09:19:59 on dist-test-slave-4wm5"
server_key: "cba721ec6695243005f17479524e51b1"
server_key_iv: "b1cb468a89aa248848d5e67be3bf0e98"
server_key_version: "encryptionkey@0"
I20260925 09:19:59.938241 6185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:19:59.975126 6185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:19:59.980530 6185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:19:59.980876 6185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:19:59.982158 6185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:19:59.983279 6045 heartbeater.cc:503] Master 127.3.82.62:44639 was elected leader, sending a full tablet report...
I20260925 09:19:59.983814 6185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:19:59.983925 6185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:19:59.984100 6185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:19:59.984200 6185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:00.023710 6185 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:35855
I20260925 09:20:00.023916 6312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:35855 every 8 connection(s)
I20260925 09:20:00.025173 6185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:00.025897 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6185
I20260925 09:20:00.026110 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:00.026680 3400 external_mini_cluster.cc:1428] Setting key e18d0bc64cbf0e1a2fdb5e5378647b9b
Sep 25 09:20:00 dist-test-slave-4wm5 krb5kdc[5811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790327999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:00.048406 6315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:00.028096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:60287 (local address 127.3.82.62:44639)
0925 09:20:00.028594 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:00.028600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:00.029885 (+ 1285us) server_negotiation.cc:407] Connection header received
0925 09:20:00.032225 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:00.032230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:00.032316 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:00.032419 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:00.034151 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:00.034938 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:00.036813 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:00.037268 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:00.037377 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:00.041469 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:00.041474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:00.041478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:00.041537 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:00.044914 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:00.045718 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:00.045722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:00.045725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:00.045962 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:00.046539 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:00.046542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:00.046545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:00.046835 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:00.046958 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:00.047797 (+ 839us) server_negotiation.cc:299] Negotiation successful
0925 09:20:00.048027 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260925 09:20:00.049369 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:00.028817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:44639 (local address 127.3.82.3:60287)
0925 09:20:00.029678 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:00.029737 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:00.031914 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:00.032655 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:00.032665 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:00.033305 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:00.033987 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:00.034008 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:00.035095 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:00.035099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:00.036614 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:00.036628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:00.037545 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:00.037554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:00.037928 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:00.038684 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:00.038710 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:00.041184 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:00.045072 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:00.045081 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:00.045096 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:00.045569 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:00.046106 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:00.046110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:00.046112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:00.046418 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:00.046954 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:00.046960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:00.047410 (+ 450us) client_negotiation.cc:769] Sending connection context
0925 09:20:00.047657 (+ 247us) client_negotiation.cc:240] Negotiation successful
0925 09:20:00.048103 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":347,"threads_started":1}
I20260925 09:20:00.050900 6313 heartbeater.cc:347] Connected to a master server at 127.3.82.62:44639
I20260925 09:20:00.051402 6313 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:00.052176 6313 heartbeater.cc:511] Master 127.3.82.62:44639 requested a full tablet report, sending...
I20260925 09:20:00.053807 5856 ts_manager.cc:194] Registered new tserver with Master: 459c112bc02c4c778d5cfbc6687ed991 (127.3.82.3:35855)
I20260925 09:20:00.055132 5856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:60287
I20260925 09:20:00.056521 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 09:20:00.089655 6315 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:00.087036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.3.82.62:44639)
0925 09:20:00.087419 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:00.087427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:00.087706 (+ 279us) server_negotiation.cc:407] Connection header received
0925 09:20:00.087945 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:00.087950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:00.088042 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:00.088179 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:00.089451 (+ 1272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52560: BlockingRecv error: recv got EOF from 127.0.0.1:52560 (error 108)
Metrics: {"server-negotiator.queue_time_us":208}
I20260925 09:20:00.093331 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5917
W20260925 09:20:00.097078 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5917: Not authorized: Client connection negotiation failed: client connection to 127.3.82.1:45695: server requires authentication, but client does not have Kerberos credentials available
I20260925 09:20:00.097294 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 5917
I20260925 09:20:00.113560 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6051
W20260925 09:20:00.116648 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6051: Not authorized: Client connection negotiation failed: client connection to 127.3.82.2:36003: server requires authentication, but client does not have Kerberos credentials available
I20260925 09:20:00.116866 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6051
W20260925 09:20:00.116927 6326 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:00.114592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.3.82.2:36003)
0925 09:20:00.115174 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:00.115181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:00.115204 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:20:00.115264 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:00.115270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:00.115531 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:00.115737 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:00.116593 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47486: BlockingRecv error: recv got EOF from 127.0.0.1:47486 (error 108)
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":255,"threads_started":1}
I20260925 09:20:00.134357 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6185
W20260925 09:20:00.138031 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6185: Not authorized: Client connection negotiation failed: client connection to 127.3.82.3:35855: server requires authentication, but client does not have Kerberos credentials available
I20260925 09:20:00.138309 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6185
W20260925 09:20:00.138403 6327 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:00.135574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.3.82.3:35855)
0925 09:20:00.136344 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:00.136351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:00.136388 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:20:00.136457 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:00.136464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:00.136752 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:00.136977 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:00.137952 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47148: BlockingRecv error: recv got EOF from 127.0.0.1:47148 (error 108)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":372,"threads_started":1}
I20260925 09:20:00.155959 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 5824
I20260925 09:20:00.157451 5886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:00.269088 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 5824
2026-09-25T09:20:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3820 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:00 dist-test-slave-4wm5 krb5kdc[6331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:00 dist-test-slave-4wm5 krb5kdc[6331](info): set up 2 sockets
Sep 25 09:20:00 dist-test-slave-4wm5 krb5kdc[6331](info): commencing operation
krb5kdc: starting...
W20260925 09:20:01.407043 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:01 dist-test-slave-4wm5 krb5kdc[6331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:01.638257 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:44137
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:44137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:01.992013 6344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:01.992420 6344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:01.992607 6344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:02.000378 6344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:02.000551 6344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:02.000669 6344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:02.000778 6344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:02.000865 6344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:02.011523 6344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:44137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:44137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:02.013022 6344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:02.014529 6344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:02.027807 6349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:02.027873 6350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:02.027845 6352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:02.028864 6344 server_base.cc:1034] running on GCE node
I20260925 09:20:02.029723 6344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:02.031517 6344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:02.032787 6344 hybrid_clock.cc:648] HybridClock initialized: now 1790328002032749 us; error 41 us; skew 500 ppm
I20260925 09:20:02.033203 6344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:02 dist-test-slave-4wm5 krb5kdc[6331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:02.050081 6344 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:02.052173 6344 webserver.cc:460] Webserver started at http://127.3.82.62:36705/ using document root <none> and password file <none>
I20260925 09:20:02.052991 6344 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:02.053175 6344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:02.053539 6344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:02.056942 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "b50ea6e782c543c59f6c6cffc628a4b9"
format_stamp: "Formatted at 2026-09-25 09:20:02 on dist-test-slave-4wm5"
server_key: "b50c48ad1f11f6d037e26a54bfa91ca6"
server_key_iv: "611be50b3840ee1dad57ea0039fd90d1"
server_key_version: "encryptionkey@0"
I20260925 09:20:02.058372 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "b50ea6e782c543c59f6c6cffc628a4b9"
format_stamp: "Formatted at 2026-09-25 09:20:02 on dist-test-slave-4wm5"
server_key: "b50c48ad1f11f6d037e26a54bfa91ca6"
server_key_iv: "611be50b3840ee1dad57ea0039fd90d1"
server_key_version: "encryptionkey@0"
I20260925 09:20:02.064949 6344 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260925 09:20:02.069542 6359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:02.071466 6344 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:02.071659 6344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "b50ea6e782c543c59f6c6cffc628a4b9"
format_stamp: "Formatted at 2026-09-25 09:20:02 on dist-test-slave-4wm5"
server_key: "b50c48ad1f11f6d037e26a54bfa91ca6"
server_key_iv: "611be50b3840ee1dad57ea0039fd90d1"
server_key_version: "encryptionkey@0"
I20260925 09:20:02.071872 6344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:02.090094 6344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:02.094965 6344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:02.095232 6344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:02.116993 6344 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:44137
I20260925 09:20:02.117029 6410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:44137 every 8 connection(s)
I20260925 09:20:02.118570 6344 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:02.120564 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6344
I20260925 09:20:02.120808 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:02.121730 3400 external_mini_cluster.cc:1428] Setting key 9f266287353bdcfa1dc8407e9583368c
I20260925 09:20:02.124372 6411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:20:02 dist-test-slave-4wm5 krb5kdc[6331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:02.139979 6411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9: Bootstrap starting.
I20260925 09:20:02.144876 6411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:02.146458 6411 log.cc:824] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:02.149461 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:02.124123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.3.82.62:44137)
0925 09:20:02.125018 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:02.125031 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:02.125073 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:20:02.126778 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:02.126808 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:02.127237 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:02.127622 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:02.129781 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:02.131336 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:02.133295 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:02.134127 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:02.134268 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:02.139591 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:02.139605 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:02.139629 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:02.139693 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:02.143995 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:02.144884 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:02.144896 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:02.144908 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:02.145162 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:02.145600 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:02.145605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:02.145608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:02.146490 (+ 882us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:02.146812 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:02.147368 (+ 556us) server_negotiation.cc:299] Negotiation successful
0925 09:20:02.147925 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":433,"threads_started":1}
I20260925 09:20:02.150465 6411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9: No bootstrap required, opened a new log
I20260925 09:20:02.154645 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:20:02.159209 6411 raft_consensus.cc:348] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44137 } }
I20260925 09:20:02.159654 6411 raft_consensus.cc:374] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:02.159791 6411 raft_consensus.cc:729] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50ea6e782c543c59f6c6cffc628a4b9, State: Initialized, Role: FOLLOWER
I20260925 09:20:02.160537 6411 consensus_queue.cc:260] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44137 } }
I20260925 09:20:02.160822 6411 raft_consensus.cc:388] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:02.161003 6411 raft_consensus.cc:482] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:02.161211 6411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:02.164994 6411 raft_consensus.cc:504] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44137 } }
I20260925 09:20:02.165731 6411 leader_election.cc:302] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b50ea6e782c543c59f6c6cffc628a4b9; no voters:
I20260925 09:20:02.166606 6411 leader_election.cc:288] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:02.166803 6417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:02.169328 6411 sys_catalog.cc:564] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:02.172257 6417 raft_consensus.cc:686] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [term 1 LEADER]: Becoming Leader. State: Replica: b50ea6e782c543c59f6c6cffc628a4b9, State: Running, Role: LEADER
I20260925 09:20:02.173313 6417 consensus_queue.cc:237] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44137 } }
I20260925 09:20:02.177287 6418 sys_catalog.cc:455] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44137 } } }
I20260925 09:20:02.177264 6419 sys_catalog.cc:455] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b50ea6e782c543c59f6c6cffc628a4b9. Latest consensus state: current_term: 1 leader_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50ea6e782c543c59f6c6cffc628a4b9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 44137 } } }
I20260925 09:20:02.177809 6419 sys_catalog.cc:458] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:02.177804 6418 sys_catalog.cc:458] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:02.178874 6427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:02.197379 6427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:02.210040 6437 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:20:02.211318 6427 catalog_manager.cc:1347] Generated new cluster ID: 2b1dcc64454a475598a896ed99917941
I20260925 09:20:02.211457 6427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:02.222415 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:02.211781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.3.82.62:44137)
0925 09:20:02.212006 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:02.212013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:02.212725 (+ 712us) server_negotiation.cc:407] Connection header received
0925 09:20:02.212946 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:02.212952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:02.213048 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:02.213181 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:02.214588 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:02.215796 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:02.217935 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:02.218652 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:02.218799 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:02.220217 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:02.220224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:02.220230 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:02.221252 (+ 1022us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:02.221345 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:02.221379 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:02.221503 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:02.221784 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 09:20:02.221924 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 09:20:02.225505 6427 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:02.227566 6427 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:02.238369 6427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b50ea6e782c543c59f6c6cffc628a4b9: Generated new TSK 0
I20260925 09:20:02.239591 6427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:20:02.251442 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6344
I20260925 09:20:02.252837 6406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:02.347062 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6344
2026-09-25T09:20:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2072 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:02 dist-test-slave-4wm5 krb5kdc[6445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:02 dist-test-slave-4wm5 krb5kdc[6445](info): set up 2 sockets
Sep 25 09:20:02 dist-test-slave-4wm5 krb5kdc[6445](info): commencing operation
krb5kdc: starting...
W20260925 09:20:03.470494 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.007s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:03 dist-test-slave-4wm5 krb5kdc[6445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:03.693696 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36197
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:04.035825 6458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:04.036164 6458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:04.036266 6458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:04.044066 6458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:04.044199 6458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:04.044247 6458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:04.044301 6458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:04.044370 6458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:04.055613 6458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:04.057102 6458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:04.058619 6458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:04.072160 6464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:04.072245 6466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:04.072187 6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:04.073637 6458 server_base.cc:1034] running on GCE node
I20260925 09:20:04.074668 6458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:04.076205 6458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:04.077458 6458 hybrid_clock.cc:648] HybridClock initialized: now 1790328004077399 us; error 45 us; skew 500 ppm
I20260925 09:20:04.077883 6458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:04 dist-test-slave-4wm5 krb5kdc[6445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328004, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:04.081775 6458 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:04.083796 6458 webserver.cc:460] Webserver started at http://127.3.82.62:37451/ using document root <none> and password file <none>
I20260925 09:20:04.084549 6458 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:04.084656 6458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:04.085037 6458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:04.088393 6458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "0f3d8737078941ef8da0714a91ee66f2"
format_stamp: "Formatted at 2026-09-25 09:20:04 on dist-test-slave-4wm5"
server_key: "f53c2770e62b1659408da45a72e219ad"
server_key_iv: "981e7a488118ad957f9736518e05b9d1"
server_key_version: "encryptionkey@0"
I20260925 09:20:04.089252 6458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "0f3d8737078941ef8da0714a91ee66f2"
format_stamp: "Formatted at 2026-09-25 09:20:04 on dist-test-slave-4wm5"
server_key: "f53c2770e62b1659408da45a72e219ad"
server_key_iv: "981e7a488118ad957f9736518e05b9d1"
server_key_version: "encryptionkey@0"
I20260925 09:20:04.095125 6458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:20:04.099254 6473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:04.100764 6458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260925 09:20:04.100967 6458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "0f3d8737078941ef8da0714a91ee66f2"
format_stamp: "Formatted at 2026-09-25 09:20:04 on dist-test-slave-4wm5"
server_key: "f53c2770e62b1659408da45a72e219ad"
server_key_iv: "981e7a488118ad957f9736518e05b9d1"
server_key_version: "encryptionkey@0"
I20260925 09:20:04.101174 6458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:04.139721 6458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:04.145463 6458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:04.145896 6458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:04.167763 6458 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36197
I20260925 09:20:04.167799 6524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36197 every 8 connection(s)
I20260925 09:20:04.169260 6458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:04.173385 6525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:04.175872 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6458
I20260925 09:20:04.176103 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:04.176939 3400 external_mini_cluster.cc:1428] Setting key df160d5acc013c736aa78e7058c83387
I20260925 09:20:04.187079 6525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2: Bootstrap starting.
I20260925 09:20:04.190550 6525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:04.191605 6525 log.cc:824] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:20:04 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:04.195719 6525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2: No bootstrap required, opened a new log
I20260925 09:20:04.202989 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:04.179792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.3.82.62:36197)
0925 09:20:04.180782 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:04.180800 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:04.180846 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:20:04.182678 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:04.182706 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:04.183125 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:04.183582 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:04.185088 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:04.186560 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:04.187916 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:04.188638 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:04.188861 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:04.193467 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:04.193478 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:04.193500 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:04.193550 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:04.197603 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:04.198580 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:04.198595 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:04.198604 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:04.198844 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:04.199313 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:04.199320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:04.199324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:04.200091 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:04.200453 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:04.200849 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 09:20:04.201355 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":417,"threads_started":1}
I20260925 09:20:04.206579 6525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3d8737078941ef8da0714a91ee66f2" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36197 } }
I20260925 09:20:04.206962 6525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:04.207104 6525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3d8737078941ef8da0714a91ee66f2, State: Initialized, Role: FOLLOWER
I20260925 09:20:04.207729 6525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3d8737078941ef8da0714a91ee66f2" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36197 } }
I20260925 09:20:04.207963 6525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:04.208113 6525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:04.208297 6525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:04.210618 6525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3d8737078941ef8da0714a91ee66f2" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36197 } }
I20260925 09:20:04.211174 6525 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3d8737078941ef8da0714a91ee66f2; no voters:
I20260925 09:20:04.211898 6525 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:04.212025 6530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:04.213011 6530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [term 1 LEADER]: Becoming Leader. State: Replica: 0f3d8737078941ef8da0714a91ee66f2, State: Running, Role: LEADER
I20260925 09:20:04.213577 6530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3d8737078941ef8da0714a91ee66f2" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36197 } }
I20260925 09:20:04.214187 6525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:04.216565 6531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3d8737078941ef8da0714a91ee66f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3d8737078941ef8da0714a91ee66f2" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36197 } } }
I20260925 09:20:04.217583 6531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:04.218434 6532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3d8737078941ef8da0714a91ee66f2. Latest consensus state: current_term: 1 leader_uuid: "0f3d8737078941ef8da0714a91ee66f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3d8737078941ef8da0714a91ee66f2" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36197 } } }
I20260925 09:20:04.218739 6532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:04.225138 6539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:04.230476 6539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:04.240418 6539 catalog_manager.cc:1347] Generated new cluster ID: fce225af65094bdbae2778de29cd9ad6
I20260925 09:20:04.240574 6539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:04.253690 6539 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:04.255479 6539 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:04.263037 6539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3d8737078941ef8da0714a91ee66f2: Generated new TSK 0
I20260925 09:20:04.263969 6539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:20:04.318585 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36197
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:04.670835 6551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:04.671144 6551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:04.671286 6551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:04.678941 6551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:04.679075 6551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:04.679253 6551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:04.689818 6551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:04.691277 6551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:04.692813 6551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:04.710506 6556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:04.711340 6557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:04.711514 6559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:04.712404 6551 server_base.cc:1034] running on GCE node
I20260925 09:20:04.713088 6551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:04.714207 6551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:04.715412 6551 hybrid_clock.cc:648] HybridClock initialized: now 1790328004715365 us; error 39 us; skew 500 ppm
I20260925 09:20:04.715801 6551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:04 dist-test-slave-4wm5 krb5kdc[6445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328004, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:04.719338 6551 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:20:04.720911 6551 webserver.cc:460] Webserver started at http://127.3.82.1:46369/ using document root <none> and password file <none>
I20260925 09:20:04.721661 6551 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:04.721788 6551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:04.722152 6551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:04.725432 6551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "43d4aaebf30d411089e6d6416c8c4c23"
format_stamp: "Formatted at 2026-09-25 09:20:04 on dist-test-slave-4wm5"
server_key: "08ca79b8f66c239fa57b1bc7167031a3"
server_key_iv: "b793a2cc7ca5f098e509fd4e7bb16452"
server_key_version: "encryptionkey@0"
I20260925 09:20:04.726327 6551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "43d4aaebf30d411089e6d6416c8c4c23"
format_stamp: "Formatted at 2026-09-25 09:20:04 on dist-test-slave-4wm5"
server_key: "08ca79b8f66c239fa57b1bc7167031a3"
server_key_iv: "b793a2cc7ca5f098e509fd4e7bb16452"
server_key_version: "encryptionkey@0"
I20260925 09:20:04.731968 6551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260925 09:20:04.736132 6566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:04.737788 6551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:20:04.738091 6551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "43d4aaebf30d411089e6d6416c8c4c23"
format_stamp: "Formatted at 2026-09-25 09:20:04 on dist-test-slave-4wm5"
server_key: "08ca79b8f66c239fa57b1bc7167031a3"
server_key_iv: "b793a2cc7ca5f098e509fd4e7bb16452"
server_key_version: "encryptionkey@0"
I20260925 09:20:04.738539 6551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:04.763604 6551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:04.768491 6551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:04.768819 6551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:04.769917 6551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:04.771355 6551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:04.771473 6551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:04.771625 6551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:04.771706 6551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:04.807909 6551 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:35309
I20260925 09:20:04.807968 6678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:35309 every 8 connection(s)
I20260925 09:20:04.809392 6551 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:04.810216 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6551
I20260925 09:20:04.810470 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:04.811285 3400 external_mini_cluster.cc:1428] Setting key 22e05392dc4609b58f5131ed3c5a1b89
Sep 25 09:20:04 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328004, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:20:04.833976 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:04.812629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:34297 (local address 127.3.82.62:36197)
0925 09:20:04.813210 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:04.813216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:04.814357 (+ 1141us) server_negotiation.cc:407] Connection header received
0925 09:20:04.816295 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:04.816304 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:04.816401 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:04.816528 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:04.818212 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:04.819204 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:04.822042 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:04.822554 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:04.822681 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:04.827526 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:04.827532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:04.827536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:04.827580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:04.830509 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:04.831378 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:04.831382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:04.831385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:04.831553 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:04.832051 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:04.832057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:04.832060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:04.832348 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:04.832514 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:04.833306 (+ 792us) server_negotiation.cc:299] Negotiation successful
0925 09:20:04.833552 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":307,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:20:04.835161 6682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:04.813189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36197 (local address 127.3.82.1:34297)
0925 09:20:04.814177 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:04.814231 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:04.815982 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:04.816786 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:04.816796 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:04.817337 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:04.818027 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:04.818046 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:04.819392 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:04.819398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:04.821799 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:04.821852 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:04.822840 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:04.822849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:04.823204 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:04.824074 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:04.824134 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:04.827139 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:04.830697 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:04.830707 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:04.830728 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:04.831219 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:04.831679 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:04.831684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:04.831687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:04.831916 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:04.832521 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:04.832532 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:04.832908 (+ 376us) client_negotiation.cc:769] Sending connection context
0925 09:20:04.833201 (+ 293us) client_negotiation.cc:240] Negotiation successful
0925 09:20:04.833938 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":383,"threads_started":1}
I20260925 09:20:04.837097 6679 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36197
I20260925 09:20:04.837507 6679 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:04.838376 6679 heartbeater.cc:511] Master 127.3.82.62:36197 requested a full tablet report, sending...
I20260925 09:20:04.840647 6490 ts_manager.cc:194] Registered new tserver with Master: 43d4aaebf30d411089e6d6416c8c4c23 (127.3.82.1:35309)
I20260925 09:20:04.842314 6490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:34297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:20:04.862489 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36197
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:05.225672 6685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:05.226107 6685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:05.226270 6685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:05.233793 6685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:05.233995 6685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:05.234174 6685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:05.245180 6685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:05.246809 6685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:05.248198 6685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:05.263144 6690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:05.263118 6691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:05.263617 6693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:05.264206 6685 server_base.cc:1034] running on GCE node
I20260925 09:20:05.264932 6685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:05.266153 6685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:05.267428 6685 hybrid_clock.cc:648] HybridClock initialized: now 1790328005267349 us; error 71 us; skew 500 ppm
I20260925 09:20:05.267827 6685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:05 dist-test-slave-4wm5 krb5kdc[6445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:05.271898 6685 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:20:05.273976 6685 webserver.cc:460] Webserver started at http://127.3.82.2:44879/ using document root <none> and password file <none>
I20260925 09:20:05.274850 6685 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:05.275029 6685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:05.275409 6685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:05.279044 6685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "0337687e9d5747bea193c6b6d5161336"
format_stamp: "Formatted at 2026-09-25 09:20:05 on dist-test-slave-4wm5"
server_key: "5214033dfda7ff3ebda07254e34dd396"
server_key_iv: "05e85a4b2220c7c8593365c1468dd35a"
server_key_version: "encryptionkey@0"
I20260925 09:20:05.279979 6685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "0337687e9d5747bea193c6b6d5161336"
format_stamp: "Formatted at 2026-09-25 09:20:05 on dist-test-slave-4wm5"
server_key: "5214033dfda7ff3ebda07254e34dd396"
server_key_iv: "05e85a4b2220c7c8593365c1468dd35a"
server_key_version: "encryptionkey@0"
I20260925 09:20:05.286391 6685 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:20:05.290586 6700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:05.292375 6685 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:20:05.292618 6685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "0337687e9d5747bea193c6b6d5161336"
format_stamp: "Formatted at 2026-09-25 09:20:05 on dist-test-slave-4wm5"
server_key: "5214033dfda7ff3ebda07254e34dd396"
server_key_iv: "05e85a4b2220c7c8593365c1468dd35a"
server_key_version: "encryptionkey@0"
I20260925 09:20:05.292879 6685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:05.334751 6685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:05.339720 6685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:05.340060 6685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:05.341123 6685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:05.342835 6685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:05.342937 6685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:05.343118 6685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:05.343204 6685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:05.378544 6685 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:41457
I20260925 09:20:05.378587 6812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:41457 every 8 connection(s)
I20260925 09:20:05.379956 6685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:05.386024 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6685
I20260925 09:20:05.386219 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:05.386802 3400 external_mini_cluster.cc:1428] Setting key 783e2917d78dd514978a587ec967f9bc
Sep 25 09:20:05 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:05.405812 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:05.383280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:37149 (local address 127.3.82.62:36197)
0925 09:20:05.383804 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:05.383808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:05.384754 (+ 946us) server_negotiation.cc:407] Connection header received
0925 09:20:05.387417 (+ 2663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:05.387424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:05.387524 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:05.387662 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:05.390210 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.390990 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.393652 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.394184 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.394316 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:05.398664 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:05.398674 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:05.398682 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:05.398737 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:05.402121 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:05.402951 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:05.402955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:05.402958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:05.403153 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:05.403771 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:05.403774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:05.403777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:05.404054 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:05.404206 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:05.405283 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0925 09:20:05.405518 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":300,"threads_started":1}
I20260925 09:20:05.407531 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:05.383708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36197 (local address 127.3.82.2:37149)
0925 09:20:05.384571 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:05.384611 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:05.387004 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:05.388304 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:05.388315 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:05.389078 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:05.390009 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:05.390039 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.391178 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.391184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:05.393469 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:05.393489 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.394438 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.394447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:05.394802 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:05.395625 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:05.395653 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:05.398388 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:05.402304 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:05.402313 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:05.402328 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:05.402800 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:05.403305 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:05.403313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:05.403318 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:05.403601 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:05.404214 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:05.404226 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:05.404813 (+ 587us) client_negotiation.cc:769] Sending connection context
0925 09:20:05.405180 (+ 367us) client_negotiation.cc:240] Negotiation successful
0925 09:20:05.405863 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":298,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:20:05.409262 6813 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36197
I20260925 09:20:05.409638 6813 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:05.410648 6813 heartbeater.cc:511] Master 127.3.82.62:36197 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:20:05.412724 6490 ts_manager.cc:194] Registered new tserver with Master: 0337687e9d5747bea193c6b6d5161336 (127.3.82.2:41457)
I20260925 09:20:05.414083 6490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:37149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:20:05.438433 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36197
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:05.780560 6819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:05.780885 6819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:05.781069 6819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:05.788596 6819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:05.788725 6819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:05.788923 6819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:05.799458 6819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:05.800895 6819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:05.802343 6819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:05.818195 6824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:05.818236 6827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:05.818203 6825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:05.819096 6819 server_base.cc:1034] running on GCE node
I20260925 09:20:05.819794 6819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:05.820890 6819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:05.822155 6819 hybrid_clock.cc:648] HybridClock initialized: now 1790328005822047 us; error 90 us; skew 500 ppm
I20260925 09:20:05.822723 6819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:05 dist-test-slave-4wm5 krb5kdc[6445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:05.826537 6819 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:20:05.828142 6819 webserver.cc:460] Webserver started at http://127.3.82.3:38619/ using document root <none> and password file <none>
I20260925 09:20:05.828864 6819 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:05.828969 6819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:05.829276 6819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:05.832635 6819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "092be9f4779843f59b97e29175b6b010"
format_stamp: "Formatted at 2026-09-25 09:20:05 on dist-test-slave-4wm5"
server_key: "ea0d68365b410380435aa18c7baa3244"
server_key_iv: "2dbd3daa2eb87bc14bc8242866afcabe"
server_key_version: "encryptionkey@0"
I20260925 09:20:05.833424 6819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "092be9f4779843f59b97e29175b6b010"
format_stamp: "Formatted at 2026-09-25 09:20:05 on dist-test-slave-4wm5"
server_key: "ea0d68365b410380435aa18c7baa3244"
server_key_iv: "2dbd3daa2eb87bc14bc8242866afcabe"
server_key_version: "encryptionkey@0"
I20260925 09:20:05.839051 6819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 09:20:05.843081 6834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:05.844839 6819 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:20:05.845064 6819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "092be9f4779843f59b97e29175b6b010"
format_stamp: "Formatted at 2026-09-25 09:20:05 on dist-test-slave-4wm5"
server_key: "ea0d68365b410380435aa18c7baa3244"
server_key_iv: "2dbd3daa2eb87bc14bc8242866afcabe"
server_key_version: "encryptionkey@0"
I20260925 09:20:05.845306 6819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:05.846614 6679 heartbeater.cc:503] Master 127.3.82.62:36197 was elected leader, sending a full tablet report...
I20260925 09:20:05.868750 6819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:05.873458 6819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:05.873739 6819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:05.874840 6819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:05.876268 6819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:05.876345 6819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:05.876434 6819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:05.876477 6819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:05.911942 6819 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:34547
I20260925 09:20:05.912045 6946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:34547 every 8 connection(s)
I20260925 09:20:05.913232 6819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:05.920778 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6819
I20260925 09:20:05.921010 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:05.921456 3400 external_mini_cluster.cc:1428] Setting key c027421c716b29aa69708ba65180186e
Sep 25 09:20:05 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:05.935542 6949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:05.916094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:44339 (local address 127.3.82.62:36197)
0925 09:20:05.916571 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:05.916577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:05.917542 (+ 965us) server_negotiation.cc:407] Connection header received
0925 09:20:05.919527 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:05.919532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:05.919633 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:05.919735 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:05.921563 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.922641 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.925066 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.925514 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.925615 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:05.929428 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:05.929434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:05.929437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:05.929478 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:05.932211 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:05.933045 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:05.933049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:05.933052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:05.933265 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:05.933696 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:05.933700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:05.933702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:05.934070 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:05.934268 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:05.934999 (+ 731us) server_negotiation.cc:299] Negotiation successful
0925 09:20:05.935256 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":208,"threads_started":1}
I20260925 09:20:05.936839 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:05.916625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36197 (local address 127.3.82.3:44339)
0925 09:20:05.917374 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:05.917412 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:05.919239 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:05.919994 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:05.920003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:05.920624 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:05.921365 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:05.921383 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.922850 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.922859 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:05.924907 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:05.924923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.925767 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.925772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:05.926087 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:05.926755 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:05.926785 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:05.929131 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:05.932374 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:05.932381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:05.932397 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:05.932908 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:05.933379 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:05.933382 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:05.933384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:05.933573 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:05.934268 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:05.934274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:05.934617 (+ 343us) client_negotiation.cc:769] Sending connection context
0925 09:20:05.934884 (+ 267us) client_negotiation.cc:240] Negotiation successful
0925 09:20:05.935505 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":272,"threads_started":1}
I20260925 09:20:05.938423 6947 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36197
I20260925 09:20:05.938769 6947 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:05.939683 6947 heartbeater.cc:511] Master 127.3.82.62:36197 requested a full tablet report, sending...
I20260925 09:20:05.941501 6490 ts_manager.cc:194] Registered new tserver with Master: 092be9f4779843f59b97e29175b6b010 (127.3.82.3:34547)
I20260925 09:20:05.942456 6490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:44339
I20260925 09:20:05.950917 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:20:05 dist-test-slave-4wm5 krb5kdc[6445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 09:20:05 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:06.003249 6949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:05.987893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42044 (local address 127.3.82.62:36197)
0925 09:20:05.988200 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:05.988211 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:05.988602 (+ 391us) server_negotiation.cc:407] Connection header received
0925 09:20:05.988909 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:05.988913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:05.988983 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:05.989103 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:05.990653 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.991423 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.992780 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:05.993230 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:05.993392 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:05.997697 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:05.997707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:05.997711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:05.997762 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:06.000635 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.001374 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.001378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.001380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.001503 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.001931 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.001936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.001939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.002275 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:06.002461 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:06.002748 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 09:20:06.002928 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
W20260925 09:20:06.003958 6490 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42044
I20260925 09:20:06.006027 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6551
Sep 25 09:20:06 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:06.024469 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:06.007281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.3.82.1:35309)
0925 09:20:06.007984 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:06.007992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:06.008019 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:20:06.008085 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:06.008092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:06.008306 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:06.008466 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:06.009951 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:06.011232 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:06.012566 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:06.013153 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:06.013259 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:06.017955 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:06.017973 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:06.017988 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:06.018030 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:06.021354 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.022214 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.022238 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.022240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.022389 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.022866 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.022871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.022873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.023294 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:06.023528 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:06.023810 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 09:20:06.024075 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":364,"threads_started":1}
W20260925 09:20:06.025976 6674 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47726
W20260925 09:20:06.027185 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6551: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:20:06.027447 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6551
I20260925 09:20:06.043476 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6685
Sep 25 09:20:06 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:20:06.062457 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:06.044538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50868 (local address 127.3.82.2:41457)
0925 09:20:06.045149 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:06.045156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:06.045183 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:20:06.045252 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:06.045258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:06.045482 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:06.045610 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:06.047087 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:06.048326 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:06.049799 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:06.050584 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:06.050765 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:06.055526 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:06.055552 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:06.055568 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:06.055617 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:06.059076 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.059929 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.059939 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.059945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.060101 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.060529 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.060536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.060540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.060940 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:06.061294 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:06.061672 (+ 378us) server_negotiation.cc:299] Negotiation successful
0925 09:20:06.062031 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":316,"threads_started":1}
W20260925 09:20:06.063812 6808 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50868
W20260925 09:20:06.065014 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6685: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:20:06.065186 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6685
I20260925 09:20:06.081231 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6819
Sep 25 09:20:06 dist-test-slave-4wm5 krb5kdc[6445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328005, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:20:06.101140 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:06.082418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.3.82.3:34547)
0925 09:20:06.083081 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:06.083089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:06.083118 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:20:06.083184 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:06.083192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:06.083522 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:06.083736 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:06.085369 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:06.086762 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:06.088171 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:06.088855 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:06.089021 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:06.094258 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:06.094276 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:06.094293 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:06.094345 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:06.098022 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.098804 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.098813 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.098815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.098964 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:06.099386 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:06.099391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:06.099394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:06.099804 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:06.100160 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:06.100454 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 09:20:06.100762 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":319,"threads_started":1}
W20260925 09:20:06.102821 6942 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60036
W20260925 09:20:06.104099 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6819: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:20:06.104346 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6819
I20260925 09:20:06.120083 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6458
I20260925 09:20:06.121459 6520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:06.230289 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6458
2026-09-25T09:20:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3889 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:06 dist-test-slave-4wm5 krb5kdc[6965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:06 dist-test-slave-4wm5 krb5kdc[6965](info): set up 2 sockets
Sep 25 09:20:06 dist-test-slave-4wm5 krb5kdc[6965](info): commencing operation
krb5kdc: starting...
W20260925 09:20:07.342516 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:07 dist-test-slave-4wm5 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:07.561765 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:34751
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:34751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:07.909179 6978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:07.909497 6978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:07.909655 6978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:07.917177 6978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:07.917304 6978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:07.917353 6978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:07.917469 6978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:07.917533 6978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:07.928017 6978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:34751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:34751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:07.929451 6978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:07.930869 6978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:07.944018 6983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:07.944067 6986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:07.944092 6984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:07.944954 6978 server_base.cc:1034] running on GCE node
I20260925 09:20:07.945897 6978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:07.947523 6978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:07.948853 6978 hybrid_clock.cc:648] HybridClock initialized: now 1790328007948746 us; error 62 us; skew 500 ppm
I20260925 09:20:07.949268 6978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:07 dist-test-slave-4wm5 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328007, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:07.957999 6978 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:07.959883 6978 webserver.cc:460] Webserver started at http://127.3.82.62:38661/ using document root <none> and password file <none>
I20260925 09:20:07.960664 6978 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:07.960832 6978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:07.961189 6978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:07.964457 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "52e9dba46a9b4250aa6f168463bc161f"
format_stamp: "Formatted at 2026-09-25 09:20:07 on dist-test-slave-4wm5"
server_key: "7b006d1e56add1223259fbf4012e17dc"
server_key_iv: "9548799b275c8449012a6a5906428701"
server_key_version: "encryptionkey@0"
I20260925 09:20:07.965372 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "52e9dba46a9b4250aa6f168463bc161f"
format_stamp: "Formatted at 2026-09-25 09:20:07 on dist-test-slave-4wm5"
server_key: "7b006d1e56add1223259fbf4012e17dc"
server_key_iv: "9548799b275c8449012a6a5906428701"
server_key_version: "encryptionkey@0"
I20260925 09:20:07.971240 6978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260925 09:20:07.975211 6993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:07.977028 6978 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260925 09:20:07.977300 6978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "52e9dba46a9b4250aa6f168463bc161f"
format_stamp: "Formatted at 2026-09-25 09:20:07 on dist-test-slave-4wm5"
server_key: "7b006d1e56add1223259fbf4012e17dc"
server_key_iv: "9548799b275c8449012a6a5906428701"
server_key_version: "encryptionkey@0"
I20260925 09:20:07.977558 6978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:08.011129 6978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:08.016533 6978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:08.016893 6978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:08.039450 6978 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:34751
I20260925 09:20:08.039500 7044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:34751 every 8 connection(s)
I20260925 09:20:08.041038 6978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:08.044114 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 6978
I20260925 09:20:08.044371 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:08.045210 3400 external_mini_cluster.cc:1428] Setting key 512a47347c87fb081873d1de2b043df6
I20260925 09:20:08.045961 7045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:08.062457 7045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f: Bootstrap starting.
Sep 25 09:20:08 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:08.066224 7045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:08.067478 7045 log.cc:824] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:08.070729 7045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f: No bootstrap required, opened a new log
I20260925 09:20:08.072331 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:08.047688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39678 (local address 127.3.82.62:34751)
0925 09:20:08.048611 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:08.048625 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:08.048673 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:20:08.051378 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:08.051412 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:08.051944 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:08.052420 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:08.054509 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:08.055962 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:08.057926 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:08.058848 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:08.059066 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:08.064111 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:08.064120 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:08.064136 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:08.064191 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:08.067762 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:08.068646 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:08.068654 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:08.068661 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:08.068873 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:08.069315 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:08.069323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:08.069326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:08.069985 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:08.070288 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:08.070806 (+ 518us) server_negotiation.cc:299] Negotiation successful
0925 09:20:08.071289 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":417,"threads_started":1}
I20260925 09:20:08.080137 7045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e9dba46a9b4250aa6f168463bc161f" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34751 } }
I20260925 09:20:08.080686 7045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:08.080788 7045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e9dba46a9b4250aa6f168463bc161f, State: Initialized, Role: FOLLOWER
I20260925 09:20:08.081310 7045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e9dba46a9b4250aa6f168463bc161f" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34751 } }
I20260925 09:20:08.081595 7045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:08.081722 7045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:08.081946 7045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:08.084249 7045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e9dba46a9b4250aa6f168463bc161f" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34751 } }
I20260925 09:20:08.084822 7045 leader_election.cc:302] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52e9dba46a9b4250aa6f168463bc161f; no voters:
I20260925 09:20:08.085503 7045 leader_election.cc:288] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:08.085660 7050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:08.086392 7050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [term 1 LEADER]: Becoming Leader. State: Replica: 52e9dba46a9b4250aa6f168463bc161f, State: Running, Role: LEADER
I20260925 09:20:08.086938 7050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e9dba46a9b4250aa6f168463bc161f" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34751 } }
I20260925 09:20:08.087770 7045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:08.090111 7052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52e9dba46a9b4250aa6f168463bc161f. Latest consensus state: current_term: 1 leader_uuid: "52e9dba46a9b4250aa6f168463bc161f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e9dba46a9b4250aa6f168463bc161f" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34751 } } }
I20260925 09:20:08.089942 7051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52e9dba46a9b4250aa6f168463bc161f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52e9dba46a9b4250aa6f168463bc161f" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34751 } } }
I20260925 09:20:08.090525 7052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:08.090525 7051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:08.092594 7055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:08.099228 7055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:08.109611 7055 catalog_manager.cc:1347] Generated new cluster ID: 769d67cd605446c78cdff27e2832ff90
I20260925 09:20:08.109750 7055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:08.123848 7055 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:08.125315 7055 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:08.134156 7055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52e9dba46a9b4250aa6f168463bc161f: Generated new TSK 0
I20260925 09:20:08.135154 7055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:20:08.199497 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34751
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:08.555845 7071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:08.556139 7071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:08.556298 7071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:08.564014 7071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:08.564137 7071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:08.564309 7071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:08.574990 7071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:08.576342 7071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:08.577665 7071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:08.593088 7076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:08.593041 7079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:08.593114 7077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:08.594225 7071 server_base.cc:1034] running on GCE node
I20260925 09:20:08.594977 7071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:08.596110 7071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:08.597352 7071 hybrid_clock.cc:648] HybridClock initialized: now 1790328008597280 us; error 67 us; skew 500 ppm
I20260925 09:20:08.597723 7071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:08 dist-test-slave-4wm5 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:08.601095 7071 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:20:08.602820 7071 webserver.cc:460] Webserver started at http://127.3.82.1:37605/ using document root <none> and password file <none>
I20260925 09:20:08.603597 7071 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:08.603753 7071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:08.604130 7071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:08.607786 7071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "0348f8b671e84b28844cf58145425d71"
format_stamp: "Formatted at 2026-09-25 09:20:08 on dist-test-slave-4wm5"
server_key: "39997ae617d4d9b6a1a095ca66b56504"
server_key_iv: "1022fae5dd2a36c4f54a4d22ad552e5b"
server_key_version: "encryptionkey@0"
I20260925 09:20:08.608590 7071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "0348f8b671e84b28844cf58145425d71"
format_stamp: "Formatted at 2026-09-25 09:20:08 on dist-test-slave-4wm5"
server_key: "39997ae617d4d9b6a1a095ca66b56504"
server_key_iv: "1022fae5dd2a36c4f54a4d22ad552e5b"
server_key_version: "encryptionkey@0"
I20260925 09:20:08.614315 7071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 09:20:08.618396 7086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:08.620060 7071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:20:08.620275 7071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "0348f8b671e84b28844cf58145425d71"
format_stamp: "Formatted at 2026-09-25 09:20:08 on dist-test-slave-4wm5"
server_key: "39997ae617d4d9b6a1a095ca66b56504"
server_key_iv: "1022fae5dd2a36c4f54a4d22ad552e5b"
server_key_version: "encryptionkey@0"
I20260925 09:20:08.620512 7071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:08.638049 7071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:08.643021 7071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:08.643349 7071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:08.644461 7071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:08.646000 7071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:08.646075 7071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:08.646246 7071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:08.646302 7071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:08.681391 7071 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:46431
I20260925 09:20:08.681488 7198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:46431 every 8 connection(s)
I20260925 09:20:08.682907 7071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:08.692811 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7071
I20260925 09:20:08.693040 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:08.693771 3400 external_mini_cluster.cc:1428] Setting key 13b350cc3dfef39c8b8abfe04c9f4f2e
Sep 25 09:20:08 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:08.706110 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:08.685645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:43817 (local address 127.3.82.62:34751)
0925 09:20:08.686176 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:08.686181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:08.687331 (+ 1150us) server_negotiation.cc:407] Connection header received
0925 09:20:08.689166 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:08.689170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:08.689269 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:08.689451 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:08.691059 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:08.691779 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:08.693603 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:08.694149 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:08.694287 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:08.698072 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:08.698081 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:08.698089 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:08.698135 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:08.701863 (+ 3728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:08.702814 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:08.702822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:08.702828 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:08.703019 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:08.704066 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:08.704070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:08.704072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:08.704410 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:08.704542 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:08.705553 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0925 09:20:08.705741 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":314,"threads_started":1}
I20260925 09:20:08.707825 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:08.686350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34751 (local address 127.3.82.1:43817)
0925 09:20:08.687144 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:08.687185 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:08.688892 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:08.689655 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:08.689665 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:08.690309 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:08.690890 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:08.690907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:08.691950 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:08.691954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:08.693449 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:08.693461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:08.694393 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:08.694401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:08.694698 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:08.695516 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:08.695553 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:08.697755 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:08.702079 (+ 4324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:08.702089 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:08.702105 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:08.702639 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:08.703679 (+ 1040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:08.703683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:08.703686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:08.703934 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:08.704575 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:08.704587 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:08.705114 (+ 527us) client_negotiation.cc:769] Sending connection context
0925 09:20:08.705447 (+ 333us) client_negotiation.cc:240] Negotiation successful
0925 09:20:08.706107 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":300,"threads_started":1}
I20260925 09:20:08.709687 7199 heartbeater.cc:347] Connected to a master server at 127.3.82.62:34751
I20260925 09:20:08.710093 7199 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:08.710868 7199 heartbeater.cc:511] Master 127.3.82.62:34751 requested a full tablet report, sending...
I20260925 09:20:08.713420 7010 ts_manager.cc:194] Registered new tserver with Master: 0348f8b671e84b28844cf58145425d71 (127.3.82.1:46431)
I20260925 09:20:08.715699 7010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:43817
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:20:08.745543 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34751
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:09.092376 7205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:09.092687 7205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:09.092783 7205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:09.100504 7205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:09.100637 7205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:09.100754 7205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:09.111299 7205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:09.112634 7205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:09.114033 7205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:09.128358 7213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:09.128391 7211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:09.128443 7210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:09.128980 7205 server_base.cc:1034] running on GCE node
I20260925 09:20:09.129891 7205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:09.131230 7205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:09.132510 7205 hybrid_clock.cc:648] HybridClock initialized: now 1790328009132434 us; error 75 us; skew 500 ppm
I20260925 09:20:09.132905 7205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:09.136590 7205 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:20:09.138322 7205 webserver.cc:460] Webserver started at http://127.3.82.2:37049/ using document root <none> and password file <none>
I20260925 09:20:09.139027 7205 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:09.139171 7205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:09.139531 7205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:09.143555 7205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "3b69acf62131460caf3951f8e93e3155"
format_stamp: "Formatted at 2026-09-25 09:20:09 on dist-test-slave-4wm5"
server_key: "0c1c86f49dbf6018adedaba56d1dcdb5"
server_key_iv: "e9b939488a0a354b4faefb3f236c3862"
server_key_version: "encryptionkey@0"
I20260925 09:20:09.144791 7205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "3b69acf62131460caf3951f8e93e3155"
format_stamp: "Formatted at 2026-09-25 09:20:09 on dist-test-slave-4wm5"
server_key: "0c1c86f49dbf6018adedaba56d1dcdb5"
server_key_iv: "e9b939488a0a354b4faefb3f236c3862"
server_key_version: "encryptionkey@0"
I20260925 09:20:09.152168 7205 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260925 09:20:09.156414 7220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:09.158247 7205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:09.158469 7205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "3b69acf62131460caf3951f8e93e3155"
format_stamp: "Formatted at 2026-09-25 09:20:09 on dist-test-slave-4wm5"
server_key: "0c1c86f49dbf6018adedaba56d1dcdb5"
server_key_iv: "e9b939488a0a354b4faefb3f236c3862"
server_key_version: "encryptionkey@0"
I20260925 09:20:09.158699 7205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:09.202436 7205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:09.207461 7205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:09.207785 7205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:09.208926 7205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:09.210398 7205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:09.210477 7205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:09.210623 7205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:09.210673 7205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:09.245276 7205 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:35277
I20260925 09:20:09.245368 7332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:35277 every 8 connection(s)
I20260925 09:20:09.246677 7205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:09.248129 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7205
I20260925 09:20:09.248332 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:09.248750 3400 external_mini_cluster.cc:1428] Setting key 2636acdeb7954a3287c7818f4737e79f
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:20:09.272944 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.249643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:60133 (local address 127.3.82.62:34751)
0925 09:20:09.250182 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:09.250188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:09.251331 (+ 1143us) server_negotiation.cc:407] Connection header received
0925 09:20:09.253295 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:09.253303 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:09.253404 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:09.253529 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:09.255328 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.256560 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.259687 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.260175 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.260296 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.265655 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:09.265661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:09.265664 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:09.265705 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:09.268862 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.270064 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.270070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.270072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.270277 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.270813 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.270818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.270820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.271174 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:09.271505 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:09.272324 (+ 819us) server_negotiation.cc:299] Negotiation successful
0925 09:20:09.272562 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":278,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:20:09.274451 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.250311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34751 (local address 127.3.82.2:60133)
0925 09:20:09.251140 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:09.251193 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:09.252987 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:09.253769 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:09.253778 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:09.254491 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:09.255113 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:09.255132 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.256830 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.256836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:09.259442 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:09.259467 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.260477 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.260487 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:09.260837 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.261730 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:09.261767 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:09.265285 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:09.269063 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:09.269078 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:09.269104 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:09.269895 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:09.270414 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:09.270420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:09.270424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:09.270651 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:09.271488 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:09.271497 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:09.271906 (+ 409us) client_negotiation.cc:769] Sending connection context
0925 09:20:09.272227 (+ 321us) client_negotiation.cc:240] Negotiation successful
0925 09:20:09.272826 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":368,"threads_started":1}
I20260925 09:20:09.280082 7333 heartbeater.cc:347] Connected to a master server at 127.3.82.62:34751
I20260925 09:20:09.280541 7333 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:09.281430 7333 heartbeater.cc:511] Master 127.3.82.62:34751 requested a full tablet report, sending...
I20260925 09:20:09.283314 7010 ts_manager.cc:194] Registered new tserver with Master: 3b69acf62131460caf3951f8e93e3155 (127.3.82.2:35277)
I20260925 09:20:09.284606 7010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:60133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:20:09.301633 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34751
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:09.654662 7339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:09.655019 7339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:09.655197 7339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:09.663735 7339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:09.663904 7339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:09.664100 7339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:09.674881 7339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:09.676309 7339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:09.677693 7339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:09.692170 7347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:09.692263 7344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:09.692237 7345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:09.692977 7339 server_base.cc:1034] running on GCE node
I20260925 09:20:09.693642 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:09.694857 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:09.696123 7339 hybrid_clock.cc:648] HybridClock initialized: now 1790328009696066 us; error 55 us; skew 500 ppm
I20260925 09:20:09.696485 7339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:09.699836 7339 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:20:09.701447 7339 webserver.cc:460] Webserver started at http://127.3.82.3:33319/ using document root <none> and password file <none>
I20260925 09:20:09.702265 7339 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:09.702437 7339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:09.702777 7339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:09.705933 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "f480d9d635d44b5a900872e32b3c35a9"
format_stamp: "Formatted at 2026-09-25 09:20:09 on dist-test-slave-4wm5"
server_key: "dfa8af14b891f0ce0f6ae67898dbe982"
server_key_iv: "ad4012ea791af61733e584e5934309ea"
server_key_version: "encryptionkey@0"
I20260925 09:20:09.706779 7339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "f480d9d635d44b5a900872e32b3c35a9"
format_stamp: "Formatted at 2026-09-25 09:20:09 on dist-test-slave-4wm5"
server_key: "dfa8af14b891f0ce0f6ae67898dbe982"
server_key_iv: "ad4012ea791af61733e584e5934309ea"
server_key_version: "encryptionkey@0"
I20260925 09:20:09.712371 7339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 09:20:09.716249 7354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:09.717785 7339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:20:09.718036 7339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "f480d9d635d44b5a900872e32b3c35a9"
format_stamp: "Formatted at 2026-09-25 09:20:09 on dist-test-slave-4wm5"
server_key: "dfa8af14b891f0ce0f6ae67898dbe982"
server_key_iv: "ad4012ea791af61733e584e5934309ea"
server_key_version: "encryptionkey@0"
I20260925 09:20:09.718225 7339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:09.719081 7199 heartbeater.cc:503] Master 127.3.82.62:34751 was elected leader, sending a full tablet report...
I20260925 09:20:09.741396 7339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:09.746493 7339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:09.746811 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:09.747830 7339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:09.749364 7339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:09.749466 7339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:09.749617 7339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:09.749688 7339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:09.786521 7339 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:37163
I20260925 09:20:09.786571 7466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:37163 every 8 connection(s)
I20260925 09:20:09.788101 7339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:09.794391 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7339
I20260925 09:20:09.794588 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:09.795006 3400 external_mini_cluster.cc:1428] Setting key f582853e92bbdae42540cc52b2f1c3a8
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:09.812372 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.791294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:44649 (local address 127.3.82.62:34751)
0925 09:20:09.791877 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:09.791881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:09.792933 (+ 1052us) server_negotiation.cc:407] Connection header received
0925 09:20:09.795584 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:09.795590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:09.795738 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:09.795880 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:09.797684 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.798712 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.800513 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.800972 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.801114 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.805219 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:09.805231 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:09.805235 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:09.805276 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:09.808487 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.809532 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.809541 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.809546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.809700 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.810257 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.810265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.810269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.810599 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:09.810782 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:09.811634 (+ 852us) server_negotiation.cc:299] Negotiation successful
0925 09:20:09.811913 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":276,"threads_started":1}
I20260925 09:20:09.813561 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.791954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34751 (local address 127.3.82.3:44649)
0925 09:20:09.792743 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:09.792791 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:09.795185 (+ 2394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:09.796206 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:09.796217 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:09.796842 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:09.797492 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:09.797514 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.798893 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.798898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:09.800331 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:09.800345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.801234 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.801242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:09.801498 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.802289 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:09.802317 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:09.804909 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:09.808729 (+ 3820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:09.808742 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:09.808763 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:09.809380 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:09.809878 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:09.809888 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:09.809891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:09.810119 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:09.810752 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:09.810762 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:09.811191 (+ 429us) client_negotiation.cc:769] Sending connection context
0925 09:20:09.811512 (+ 321us) client_negotiation.cc:240] Negotiation successful
0925 09:20:09.812130 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":324,"threads_started":1}
I20260925 09:20:09.815274 7467 heartbeater.cc:347] Connected to a master server at 127.3.82.62:34751
I20260925 09:20:09.815600 7467 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:09.816521 7467 heartbeater.cc:511] Master 127.3.82.62:34751 requested a full tablet report, sending...
I20260925 09:20:09.818217 7010 ts_manager.cc:194] Registered new tserver with Master: f480d9d635d44b5a900872e32b3c35a9 (127.3.82.3:37163)
I20260925 09:20:09.819105 7010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:44649
I20260925 09:20:09.823900 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:09.879712 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.861033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.3.82.1:46431)
0925 09:20:09.861919 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:09.861931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:09.861976 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:20:09.862046 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:09.862053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:09.862421 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:09.862643 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:09.864497 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.865616 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.867071 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.867860 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.867986 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.872671 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:09.872688 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:09.872703 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:09.872754 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:09.876201 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.877094 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.877102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.877105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.877317 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.877779 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.877782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.877784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.878284 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:09.878519 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:09.879026 (+ 507us) server_negotiation.cc:299] Negotiation successful
0925 09:20:09.879334 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":354,"threads_started":1}
Sep 25 09:20:09 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:09.901110 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.885407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.3.82.62:34751)
0925 09:20:09.885652 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:09.885668 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:09.885994 (+ 326us) server_negotiation.cc:407] Connection header received
0925 09:20:09.886321 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:09.886326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:09.886414 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:09.886509 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:09.887819 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.888837 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.890372 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.890874 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.891040 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.895623 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:09.895629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:09.895632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:09.895674 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:09.898354 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.899029 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.899033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.899036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.899167 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.899616 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.899620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.899622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.899975 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:09.900162 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:09.900523 (+ 361us) server_negotiation.cc:299] Negotiation successful
0925 09:20:09.900711 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260925 09:20:09.901736 7010 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39686
I20260925 09:20:09.902912 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7071
I20260925 09:20:09.916333 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:09.904105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.3.82.1:46431)
0925 09:20:09.904317 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:09.904323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:09.904436 (+ 113us) server_negotiation.cc:407] Connection header received
0925 09:20:09.904572 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:09.904575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:09.904648 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:09.904729 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:09.906000 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.906693 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.907936 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:09.908382 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:09.908518 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:09.910933 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:09.910938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:09.910941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:09.910979 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:09.913526 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.914320 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.914328 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.914332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.914470 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:09.914886 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:09.914890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:09.914892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:09.915157 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:09.915414 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:09.915751 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 09:20:09.915996 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 09:20:09.917102 7194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:10.006265 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7071
I20260925 09:20:10.024041 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7205
Sep 25 09:20:10 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:20:10.042984 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:10.024978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.3.82.2:35277)
0925 09:20:10.025584 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:10.025591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:10.025616 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:20:10.025681 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:10.025687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:10.026003 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:10.026183 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:10.027411 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:10.028440 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:10.029767 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:10.030504 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:10.030656 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:10.035573 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:10.035593 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:10.035611 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:10.035669 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:10.039396 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:10.040486 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:10.040495 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:10.040497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:10.040648 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:10.041097 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:10.041100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:10.041102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:10.041561 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:10.041810 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:10.042207 (+ 397us) server_negotiation.cc:299] Negotiation successful
0925 09:20:10.042504 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":321,"threads_started":1}
I20260925 09:20:10.044414 7328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:10.132364 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7205
I20260925 09:20:10.148439 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7339
Sep 25 09:20:10 dist-test-slave-4wm5 krb5kdc[6965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:20:10.167950 7482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:10.149562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.3.82.3:37163)
0925 09:20:10.150226 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:10.150233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:10.150262 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:20:10.150323 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:10.150330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:10.150569 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:10.150711 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:10.152067 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:10.153148 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:10.154858 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:10.155486 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:10.155621 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:10.160755 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:10.160779 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:10.160797 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:10.160849 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:10.164518 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:10.165349 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:10.165358 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:10.165360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:10.165534 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:10.166115 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:10.166121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:10.166123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:10.166579 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:10.166889 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:10.167373 (+ 484us) server_negotiation.cc:299] Negotiation successful
0925 09:20:10.167614 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":376,"threads_started":1}
I20260925 09:20:10.169354 7462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:10.257793 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7339
I20260925 09:20:10.274013 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 6978
I20260925 09:20:10.275377 7040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:10.381681 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 6978
2026-09-25T09:20:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4152 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:10 dist-test-slave-4wm5 krb5kdc[7487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:10 dist-test-slave-4wm5 krb5kdc[7487](info): set up 2 sockets
Sep 25 09:20:10 dist-test-slave-4wm5 krb5kdc[7487](info): commencing operation
krb5kdc: starting...
W20260925 09:20:11.538604 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:11 dist-test-slave-4wm5 krb5kdc[7487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:11.759065 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45761
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:12.111049 7500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:12.111449 7500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:12.111613 7500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:12.119482 7500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:12.119644 7500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:12.119751 7500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:12.119848 7500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:12.119933 7500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:12.130595 7500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:12.131999 7500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:12.133375 7500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:12.145816 7508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:12.145869 7506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:12.146216 7505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:12.146795 7500 server_base.cc:1034] running on GCE node
I20260925 09:20:12.147666 7500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:12.149384 7500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:12.150642 7500 hybrid_clock.cc:648] HybridClock initialized: now 1790328012150594 us; error 57 us; skew 500 ppm
I20260925 09:20:12.151077 7500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:12 dist-test-slave-4wm5 krb5kdc[7487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328012, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:12.154763 7500 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:12.156036 7500 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:12.156178 7500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:12.156526 7500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:12.159976 7500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "0969db43c87e4df69c03aeddd68bce3b"
format_stamp: "Formatted at 2026-09-25 09:20:12 on dist-test-slave-4wm5"
server_key: "d70e81e9eb27e6359e13bd8805262784"
server_key_iv: "9e7dd0e1a068d320def3a434ba0c3908"
server_key_version: "encryptionkey@0"
I20260925 09:20:12.161064 7500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "0969db43c87e4df69c03aeddd68bce3b"
format_stamp: "Formatted at 2026-09-25 09:20:12 on dist-test-slave-4wm5"
server_key: "d70e81e9eb27e6359e13bd8805262784"
server_key_iv: "9e7dd0e1a068d320def3a434ba0c3908"
server_key_version: "encryptionkey@0"
I20260925 09:20:12.167492 7500 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:20:12.171793 7514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:12.173481 7500 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 09:20:12.173719 7500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "0969db43c87e4df69c03aeddd68bce3b"
format_stamp: "Formatted at 2026-09-25 09:20:12 on dist-test-slave-4wm5"
server_key: "d70e81e9eb27e6359e13bd8805262784"
server_key_iv: "9e7dd0e1a068d320def3a434ba0c3908"
server_key_version: "encryptionkey@0"
I20260925 09:20:12.174045 7500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:12.192200 7500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:12.197161 7500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:12.197471 7500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:12.216692 7500 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45761
I20260925 09:20:12.216717 7565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45761 every 8 connection(s)
I20260925 09:20:12.217797 7500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:12.220979 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7500
I20260925 09:20:12.221222 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:12.221922 3400 external_mini_cluster.cc:1428] Setting key fd24abc3c10dcc1fb43997a22f0c0dae
I20260925 09:20:12.222378 7566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:12.236486 7566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b: Bootstrap starting.
Sep 25 09:20:12 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:12.240147 7566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:12.241309 7566 log.cc:824] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:12.244416 7566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b: No bootstrap required, opened a new log
I20260925 09:20:12.247582 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:12.224165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.3.82.62:45761)
0925 09:20:12.225129 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:12.225150 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:12.225580 (+ 430us) server_negotiation.cc:407] Connection header received
0925 09:20:12.227964 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:12.227992 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:12.228572 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:12.229070 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:12.230862 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:12.232326 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:12.233975 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:12.234817 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:12.235060 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:12.239711 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:12.239721 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:12.239739 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:12.239784 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:12.243075 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:12.243880 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:12.243888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:12.243895 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:12.244068 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:12.244494 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:12.244506 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:12.244508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:12.245146 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:12.245465 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:12.245978 (+ 513us) server_negotiation.cc:299] Negotiation successful
0925 09:20:12.246473 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":443,"threads_started":1}
I20260925 09:20:12.253175 7566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0969db43c87e4df69c03aeddd68bce3b" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45761 } }
I20260925 09:20:12.253721 7566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:12.253904 7566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0969db43c87e4df69c03aeddd68bce3b, State: Initialized, Role: FOLLOWER
I20260925 09:20:12.254504 7566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0969db43c87e4df69c03aeddd68bce3b" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45761 } }
I20260925 09:20:12.254763 7566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:12.254923 7566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:12.255110 7566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:12.257475 7566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0969db43c87e4df69c03aeddd68bce3b" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45761 } }
I20260925 09:20:12.258224 7566 leader_election.cc:302] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0969db43c87e4df69c03aeddd68bce3b; no voters:
I20260925 09:20:12.259032 7566 leader_election.cc:288] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:12.259164 7571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:12.259848 7571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [term 1 LEADER]: Becoming Leader. State: Replica: 0969db43c87e4df69c03aeddd68bce3b, State: Running, Role: LEADER
I20260925 09:20:12.260447 7571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0969db43c87e4df69c03aeddd68bce3b" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45761 } }
I20260925 09:20:12.261308 7566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:12.263298 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0969db43c87e4df69c03aeddd68bce3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0969db43c87e4df69c03aeddd68bce3b" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45761 } } }
I20260925 09:20:12.263829 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:12.263859 7573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0969db43c87e4df69c03aeddd68bce3b. Latest consensus state: current_term: 1 leader_uuid: "0969db43c87e4df69c03aeddd68bce3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0969db43c87e4df69c03aeddd68bce3b" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45761 } } }
I20260925 09:20:12.264199 7573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:12.267727 7576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:12.273669 7576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:12.285557 7576 catalog_manager.cc:1347] Generated new cluster ID: a6b49a6ecf704953b341cbad4a44e6d1
I20260925 09:20:12.285709 7576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:12.307739 7576 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:12.309680 7576 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:12.324479 7576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b: Generated new TSK 0
I20260925 09:20:12.325647 7576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:20:12.386191 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45761
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:12.740646 7592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:12.741029 7592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:12.741321 7592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:12.749495 7592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:12.749698 7592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:12.749969 7592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:12.760768 7592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:12.762538 7592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:12.764037 7592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:12.779152 7598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:12.779917 7597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:12.779923 7600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:12.780901 7592 server_base.cc:1034] running on GCE node
I20260925 09:20:12.781704 7592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:12.782893 7592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:12.784211 7592 hybrid_clock.cc:648] HybridClock initialized: now 1790328012784125 us; error 82 us; skew 500 ppm
I20260925 09:20:12.784727 7592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:12 dist-test-slave-4wm5 krb5kdc[7487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328012, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:12.794492 7592 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:20:12.795889 7592 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:12.796039 7592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:12.796380 7592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:12.799554 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "190220a8090a4b7aa6d66bf0685f3151"
format_stamp: "Formatted at 2026-09-25 09:20:12 on dist-test-slave-4wm5"
server_key: "6ab011ee72a8ebdc5005ba55dd6d1263"
server_key_iv: "7f552aa0220c88f67c7343e5f017d76c"
server_key_version: "encryptionkey@0"
I20260925 09:20:12.800417 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "190220a8090a4b7aa6d66bf0685f3151"
format_stamp: "Formatted at 2026-09-25 09:20:12 on dist-test-slave-4wm5"
server_key: "6ab011ee72a8ebdc5005ba55dd6d1263"
server_key_iv: "7f552aa0220c88f67c7343e5f017d76c"
server_key_version: "encryptionkey@0"
I20260925 09:20:12.806434 7592 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:20:12.810449 7606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:12.811947 7592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:20:12.812158 7592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "190220a8090a4b7aa6d66bf0685f3151"
format_stamp: "Formatted at 2026-09-25 09:20:12 on dist-test-slave-4wm5"
server_key: "6ab011ee72a8ebdc5005ba55dd6d1263"
server_key_iv: "7f552aa0220c88f67c7343e5f017d76c"
server_key_version: "encryptionkey@0"
I20260925 09:20:12.812417 7592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:12.828873 7592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:12.833441 7592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:12.833751 7592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:12.834810 7592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:12.836208 7592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:12.836359 7592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:12.836498 7592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:12.836585 7592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:12.872071 7592 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:40525
I20260925 09:20:12.872129 7718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:40525 every 8 connection(s)
I20260925 09:20:12.872968 7592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:12.880156 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7592
I20260925 09:20:12.880385 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:12.881036 3400 external_mini_cluster.cc:1428] Setting key 409a3bc45882c1f67a2f907ff7473849
Sep 25 09:20:12 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328012, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:12.894910 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:12.875550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:34495 (local address 127.3.82.62:45761)
0925 09:20:12.875961 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:12.875966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:12.876952 (+ 986us) server_negotiation.cc:407] Connection header received
0925 09:20:12.879176 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:12.879181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:12.879315 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:12.879443 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:12.881137 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:12.882068 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:12.883893 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:12.884310 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:12.884451 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:12.888531 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:12.888536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:12.888540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:12.888581 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:12.891405 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:12.892315 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:12.892323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:12.892326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:12.892460 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:12.892937 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:12.892944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:12.892947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:12.893245 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:12.893405 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:12.894293 (+ 888us) server_negotiation.cc:299] Negotiation successful
0925 09:20:12.894538 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":202,"threads_started":1}
I20260925 09:20:12.895963 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:12.876076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45761 (local address 127.3.82.1:34495)
0925 09:20:12.876771 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:12.876831 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:12.878890 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:12.879691 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:12.879700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:12.880266 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:12.880956 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:12.880974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:12.882263 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:12.882267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:12.883721 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:12.883734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:12.884547 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:12.884553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:12.884781 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:12.885547 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:12.885569 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:12.888226 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:12.891587 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:12.891598 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:12.891618 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:12.892111 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:12.892585 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:12.892590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:12.892594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:12.892815 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:12.893381 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:12.893389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:12.893774 (+ 385us) client_negotiation.cc:769] Sending connection context
0925 09:20:12.894137 (+ 363us) client_negotiation.cc:240] Negotiation successful
0925 09:20:12.894765 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":278,"threads_started":1}
I20260925 09:20:12.897943 7719 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45761
I20260925 09:20:12.898344 7719 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:12.899046 7719 heartbeater.cc:511] Master 127.3.82.62:45761 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:20:12.901417 7531 ts_manager.cc:194] Registered new tserver with Master: 190220a8090a4b7aa6d66bf0685f3151 (127.3.82.1:40525)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:20:12.903162 7531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:34495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:20:12.929606 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45761
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:13.295054 7725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:13.295430 7725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:13.295612 7725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:13.303195 7725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:13.303402 7725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:13.303601 7725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:13.314152 7725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:13.315595 7725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:13.317226 7725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:13.330921 7733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:13.330994 7731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:13.330971 7730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:13.332391 7725 server_base.cc:1034] running on GCE node
I20260925 09:20:13.333091 7725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:13.334249 7725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:13.335520 7725 hybrid_clock.cc:648] HybridClock initialized: now 1790328013335459 us; error 89 us; skew 500 ppm
I20260925 09:20:13.335920 7725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:13 dist-test-slave-4wm5 krb5kdc[7487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:13.339542 7725 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:20:13.341231 7725 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:13.341398 7725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:13.341815 7725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:13.345422 7725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "0e60e94b5979470a99c9510be38bd497"
format_stamp: "Formatted at 2026-09-25 09:20:13 on dist-test-slave-4wm5"
server_key: "b4c3eddbb2bbeb4c5eb59883ae3504bf"
server_key_iv: "43cc99d1ac56f34dfe3a59a9db4c1a67"
server_key_version: "encryptionkey@0"
I20260925 09:20:13.346520 7725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "0e60e94b5979470a99c9510be38bd497"
format_stamp: "Formatted at 2026-09-25 09:20:13 on dist-test-slave-4wm5"
server_key: "b4c3eddbb2bbeb4c5eb59883ae3504bf"
server_key_iv: "43cc99d1ac56f34dfe3a59a9db4c1a67"
server_key_version: "encryptionkey@0"
I20260925 09:20:13.352870 7725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260925 09:20:13.357038 7739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:13.358856 7725 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260925 09:20:13.359098 7725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "0e60e94b5979470a99c9510be38bd497"
format_stamp: "Formatted at 2026-09-25 09:20:13 on dist-test-slave-4wm5"
server_key: "b4c3eddbb2bbeb4c5eb59883ae3504bf"
server_key_iv: "43cc99d1ac56f34dfe3a59a9db4c1a67"
server_key_version: "encryptionkey@0"
I20260925 09:20:13.359406 7725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:13.390582 7725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:13.395877 7725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:13.396140 7725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:13.397233 7725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:13.398800 7725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:13.398900 7725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:13.398995 7725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:13.399056 7725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:13.436735 7725 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:34027
I20260925 09:20:13.436786 7851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:34027 every 8 connection(s)
I20260925 09:20:13.437736 7725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:13.441176 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7725
I20260925 09:20:13.441462 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:13.441987 3400 external_mini_cluster.cc:1428] Setting key 9ee9c7f19891c166749fb2a9841f2e95
Sep 25 09:20:13 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:13.460559 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:13.440490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:36821 (local address 127.3.82.62:45761)
0925 09:20:13.440971 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:13.440977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:13.442121 (+ 1144us) server_negotiation.cc:407] Connection header received
0925 09:20:13.444407 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:13.444412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:13.444498 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:13.444629 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:13.446441 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.447185 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.448911 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.449358 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.449506 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:13.453268 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:13.453273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:13.453276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:13.453311 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:13.456327 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:13.457346 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:13.457353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:13.457356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:13.457511 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:13.458149 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:13.458154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:13.458157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:13.458485 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:13.458681 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:13.459577 (+ 896us) server_negotiation.cc:299] Negotiation successful
0925 09:20:13.460153 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":212,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:20:13.461954 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:13.441000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45761 (local address 127.3.82.2:36821)
0925 09:20:13.441933 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:13.441991 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:13.444119 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:13.444885 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:13.444897 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:13.445501 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:13.446239 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:13.446259 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.447357 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.447363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:13.448726 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:13.448739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.449618 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.449625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:13.449929 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:13.450685 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:13.450709 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:13.453003 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:13.456574 (+ 3571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:13.456585 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:13.456617 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:13.457159 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:13.457655 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:13.457661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:13.457665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:13.457995 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:13.458652 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:13.458662 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:13.459113 (+ 451us) client_negotiation.cc:769] Sending connection context
0925 09:20:13.459432 (+ 319us) client_negotiation.cc:240] Negotiation successful
0925 09:20:13.460378 (+ 946us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":412,"threads_started":1}
I20260925 09:20:13.463878 7852 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45761
I20260925 09:20:13.464363 7852 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:20:13.465269 7852 heartbeater.cc:511] Master 127.3.82.62:45761 requested a full tablet report, sending...
I20260925 09:20:13.467085 7531 ts_manager.cc:194] Registered new tserver with Master: 0e60e94b5979470a99c9510be38bd497 (127.3.82.2:34027)
I20260925 09:20:13.468393 7531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:36821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:20:13.495854 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45761
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:13.840721 7858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:13.841037 7858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:13.841136 7858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:13.848484 7858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:13.848616 7858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:13.848742 7858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:13.858664 7858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:13.860011 7858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:13.861351 7858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:13.877069 7864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:13.877027 7863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:13.877624 7866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:13.878330 7858 server_base.cc:1034] running on GCE node
I20260925 09:20:13.879055 7858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:13.880163 7858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:13.881484 7858 hybrid_clock.cc:648] HybridClock initialized: now 1790328013881421 us; error 69 us; skew 500 ppm
I20260925 09:20:13.881876 7858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:13 dist-test-slave-4wm5 krb5kdc[7487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:13.885270 7858 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:20:13.886686 7858 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:13.886830 7858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:13.887177 7858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:13.890311 7858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "4a5bf437e4e24e9ba208017a4bb49b23"
format_stamp: "Formatted at 2026-09-25 09:20:13 on dist-test-slave-4wm5"
server_key: "65ec0552c357c0f0e2512516c4c56c9b"
server_key_iv: "0f5c31ddb8d269bd19326430689400d0"
server_key_version: "encryptionkey@0"
I20260925 09:20:13.891458 7858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "4a5bf437e4e24e9ba208017a4bb49b23"
format_stamp: "Formatted at 2026-09-25 09:20:13 on dist-test-slave-4wm5"
server_key: "65ec0552c357c0f0e2512516c4c56c9b"
server_key_iv: "0f5c31ddb8d269bd19326430689400d0"
server_key_version: "encryptionkey@0"
I20260925 09:20:13.897243 7858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260925 09:20:13.901437 7872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:13.903168 7858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:13.903414 7858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "4a5bf437e4e24e9ba208017a4bb49b23"
format_stamp: "Formatted at 2026-09-25 09:20:13 on dist-test-slave-4wm5"
server_key: "65ec0552c357c0f0e2512516c4c56c9b"
server_key_iv: "0f5c31ddb8d269bd19326430689400d0"
server_key_version: "encryptionkey@0"
I20260925 09:20:13.903746 7858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:13.907112 7719 heartbeater.cc:503] Master 127.3.82.62:45761 was elected leader, sending a full tablet report...
I20260925 09:20:13.920998 7858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:13.925990 7858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:13.926306 7858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:13.927345 7858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:13.928862 7858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:13.928987 7858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:13.929124 7858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:13.929214 7858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:13.965015 7858 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:46123
I20260925 09:20:13.965077 7984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:46123 every 8 connection(s)
I20260925 09:20:13.966070 7858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:13.967927 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 7858
I20260925 09:20:13.968142 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:13.968607 3400 external_mini_cluster.cc:1428] Setting key 4fc62f78e97deadac87b0f3ceeef46b1
Sep 25 09:20:13 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:13.990561 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:13.969158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:36527 (local address 127.3.82.62:45761)
0925 09:20:13.969783 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:13.969789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:13.971235 (+ 1446us) server_negotiation.cc:407] Connection header received
0925 09:20:13.973339 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:13.973345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:13.973464 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:13.973598 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:13.975406 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.976197 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.978571 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.979021 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.979163 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:13.983650 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:13.983655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:13.983658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:13.983697 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:13.986979 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:13.987942 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:13.987947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:13.987949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:13.988080 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:13.988570 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:13.988574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:13.988576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:13.988874 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:13.989045 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:13.989880 (+ 835us) server_negotiation.cc:299] Negotiation successful
0925 09:20:13.990142 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":353,"threads_started":1}
I20260925 09:20:13.991595 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:13.970027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45761 (local address 127.3.82.3:36527)
0925 09:20:13.971073 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:13.971121 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:13.973033 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:13.973947 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:13.973957 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:13.974551 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:13.975196 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:13.975215 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.976380 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.976384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:13.978404 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:13.978419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:13.979288 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:13.979297 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:13.979545 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:13.980314 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:13.980343 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:13.983377 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:13.987157 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:13.987169 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:13.987197 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:13.987775 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:13.988203 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:13.988208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:13.988212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:13.988430 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:13.989018 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:13.989027 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:13.989428 (+ 401us) client_negotiation.cc:769] Sending connection context
0925 09:20:13.989726 (+ 298us) client_negotiation.cc:240] Negotiation successful
0925 09:20:13.990347 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":498,"threads_started":1}
I20260925 09:20:13.993312 7985 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45761
I20260925 09:20:13.993620 7985 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:13.994401 7985 heartbeater.cc:511] Master 127.3.82.62:45761 requested a full tablet report, sending...
I20260925 09:20:13.996232 7531 ts_manager.cc:194] Registered new tserver with Master: 4a5bf437e4e24e9ba208017a4bb49b23 (127.3.82.3:46123)
I20260925 09:20:13.997633 7531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:36527
I20260925 09:20:13.998353 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:20:14.018712 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.003649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.3.82.62:45761)
0925 09:20:14.003979 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.003987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.004423 (+ 436us) server_negotiation.cc:407] Connection header received
0925 09:20:14.004680 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.004686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.004767 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.004935 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:14.006582 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.007336 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.008884 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.009361 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.009516 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.012130 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:14.012136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:14.012139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:14.012184 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:14.015332 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.016305 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.016316 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.016318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.016470 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.016971 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.016978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.016981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.017415 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:14.017691 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.018112 (+ 421us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.018376 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260925 09:20:14.023494 7531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:14.030730 7531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:14.077893 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.045745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.3.82.2:34027)
0925 09:20:14.067790 (+ 22045us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.067800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.067834 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:20:14.067945 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.067953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.068275 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.069448 (+ 1173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:14.069721 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.072066 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.074665 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.076379 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.076522 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.076731 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.077280 (+ 549us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.077488 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21858,"thread_start_us":21956,"threads_started":1}
I20260925 09:20:14.077970 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.044913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:34027 (local address 127.0.0.1:38026)
0925 09:20:14.067486 (+ 22573us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:14.067507 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:14.067636 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:14.068576 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:14.068581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:14.068612 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:14.069192 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.069210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.072277 (+ 3067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.072281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.074298 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.074317 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.076895 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.076901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.077078 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.077111 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:20:14.077210 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 09:20:14.077446 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22435,"mutex_wait_us":121,"thread_start_us":21136,"threads_started":1}
I20260925 09:20:14.078784 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.044630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40525 (local address 127.0.0.1:57904)
0925 09:20:14.066497 (+ 21867us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:14.066540 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:14.066749 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:14.068537 (+ 1788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:14.068545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:14.068574 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:14.069135 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.069149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.072209 (+ 3060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.072215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.075106 (+ 2891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.075115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.076543 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.076549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.076785 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.076858 (+ 73us) client_negotiation.cc:769] Sending connection context
0925 09:20:14.078295 (+ 1437us) client_negotiation.cc:240] Negotiation successful
0925 09:20:14.078442 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21580,"mutex_wait_us":125,"thread_start_us":23295,"threads_started":1}
I20260925 09:20:14.079312 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.047391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.3.82.3:46123)
0925 09:20:14.068717 (+ 21326us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.068728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.068763 (+ 35us) server_negotiation.cc:407] Connection header received
0925 09:20:14.068862 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.068869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.069276 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.069460 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:14.070583 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.071816 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.075333 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.076916 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.077043 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.078674 (+ 1631us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.078785 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.078965 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21126,"thread_start_us":21376,"threads_started":1}
I20260925 09:20:14.079702 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.047072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:46123 (local address 127.0.0.1:54846)
0925 09:20:14.068102 (+ 21030us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:14.068118 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:14.068217 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:14.069760 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:14.069767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:14.069800 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:14.070322 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.070335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.071974 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.071979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.075099 (+ 3120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.075116 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.077356 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.077362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.077530 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.077563 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:20:14.077641 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 09:20:14.077773 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20907,"mutex_wait_us":190,"thread_start_us":20076,"threads_started":1}
I20260925 09:20:14.079741 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.045174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.3.82.1:40525)
0925 09:20:14.067848 (+ 22674us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.067861 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.067894 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:20:14.068038 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.068047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.068291 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.068537 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:14.069543 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.072066 (+ 2523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.075329 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.076316 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.076484 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.076734 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.079163 (+ 2429us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.079376 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22462,"thread_start_us":22278,"threads_started":1}
I20260925 09:20:14.082602 7787 tablet_service.cc:1467] Processing CreateTablet for tablet 759e41200ecc46e9b41ba65b934c5da3 (DEFAULT_TABLE table=test-table [id=90eb6a14c92c4147a986dee2f81b0d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:14.083055 7654 tablet_service.cc:1467] Processing CreateTablet for tablet 759e41200ecc46e9b41ba65b934c5da3 (DEFAULT_TABLE table=test-table [id=90eb6a14c92c4147a986dee2f81b0d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:14.083531 7920 tablet_service.cc:1467] Processing CreateTablet for tablet 759e41200ecc46e9b41ba65b934c5da3 (DEFAULT_TABLE table=test-table [id=90eb6a14c92c4147a986dee2f81b0d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:14.084200 7787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759e41200ecc46e9b41ba65b934c5da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:14.084296 7654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759e41200ecc46e9b41ba65b934c5da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:14.084811 7920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759e41200ecc46e9b41ba65b934c5da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:14.095523 8004 tablet_bootstrap.cc:492] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Bootstrap starting.
I20260925 09:20:14.099912 8004 tablet_bootstrap.cc:654] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:14.101110 8005 tablet_bootstrap.cc:492] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Bootstrap starting.
I20260925 09:20:14.101717 8004 log.cc:824] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:14.106295 8005 tablet_bootstrap.cc:654] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:14.106706 8006 tablet_bootstrap.cc:492] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Bootstrap starting.
I20260925 09:20:14.108042 8005 log.cc:824] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:14.110525 8006 tablet_bootstrap.cc:654] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:14.111728 8006 log.cc:824] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:14.111991 8004 tablet_bootstrap.cc:492] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: No bootstrap required, opened a new log
I20260925 09:20:14.112310 8004 ts_tablet_manager.cc:1397] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260925 09:20:14.112813 8005 tablet_bootstrap.cc:492] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: No bootstrap required, opened a new log
I20260925 09:20:14.113077 8005 ts_tablet_manager.cc:1397] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260925 09:20:14.114517 8006 tablet_bootstrap.cc:492] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: No bootstrap required, opened a new log
I20260925 09:20:14.114790 8006 ts_tablet_manager.cc:1397] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260925 09:20:14.122207 8006 raft_consensus.cc:348] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.122668 8006 raft_consensus.cc:374] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:14.122764 8006 raft_consensus.cc:729] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e60e94b5979470a99c9510be38bd497, State: Initialized, Role: FOLLOWER
I20260925 09:20:14.123477 8006 consensus_queue.cc:260] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.123687 8004 raft_consensus.cc:348] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.124212 8004 raft_consensus.cc:374] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:14.124331 8004 raft_consensus.cc:729] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190220a8090a4b7aa6d66bf0685f3151, State: Initialized, Role: FOLLOWER
I20260925 09:20:14.125038 7852 heartbeater.cc:503] Master 127.3.82.62:45761 was elected leader, sending a full tablet report...
I20260925 09:20:14.124742 8005 raft_consensus.cc:348] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.125182 8005 raft_consensus.cc:374] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:14.125273 8005 raft_consensus.cc:729] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5bf437e4e24e9ba208017a4bb49b23, State: Initialized, Role: FOLLOWER
I20260925 09:20:14.125140 8004 consensus_queue.cc:260] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.125777 8006 ts_tablet_manager.cc:1428] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260925 09:20:14.126088 8005 consensus_queue.cc:260] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.127355 7985 heartbeater.cc:503] Master 127.3.82.62:45761 was elected leader, sending a full tablet report...
I20260925 09:20:14.128342 8005 ts_tablet_manager.cc:1428] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260925 09:20:14.130141 8004 ts_tablet_manager.cc:1428] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260925 09:20:14.144764 8012 raft_consensus.cc:482] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:14.145038 8012 raft_consensus.cc:504] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.146955 8012 leader_election.cc:288] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 190220a8090a4b7aa6d66bf0685f3151 (127.3.82.1:40525), 0e60e94b5979470a99c9510be38bd497 (127.3.82.2:34027)
I20260925 09:20:14.154218 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.147428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:60811 (local address 127.3.82.1:40525)
0925 09:20:14.147703 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.147713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.147744 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:20:14.147848 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.147857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.147950 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.148065 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:14.148625 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.149855 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.152158 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.153120 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.153269 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.153382 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.153729 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.153929 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:20:14.154258 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.147193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40525 (local address 127.3.82.3:60811)
0925 09:20:14.147430 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:14.147454 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:14.147599 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:14.148107 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:14.148112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:14.148142 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:14.148375 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.148386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.150009 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.150014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.151953 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.151969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.153368 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.153375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.153551 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.153580 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:20:14.153676 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 09:20:14.153790 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":6}
I20260925 09:20:14.155036 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.147661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50247 (local address 127.3.82.2:34027)
0925 09:20:14.147880 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.147888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.148427 (+ 539us) server_negotiation.cc:407] Connection header received
0925 09:20:14.148636 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.148641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.148737 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.148851 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:14.149537 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.150711 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.152755 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.153919 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.154061 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.154154 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.154635 (+ 481us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.154766 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:20:14.155130 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.147309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:34027 (local address 127.3.82.3:50247)
0925 09:20:14.148228 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:14.148246 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:14.148374 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:14.148923 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:14.148927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:14.148952 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:14.149309 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.149321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.150868 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.150874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.152589 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:14.152600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.154170 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.154177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:14.154390 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.154428 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 09:20:14.154582 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 09:20:14.154728 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":561,"threads_started":1}
I20260925 09:20:14.155443 7674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e41200ecc46e9b41ba65b934c5da3" candidate_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190220a8090a4b7aa6d66bf0685f3151" is_pre_election: true
I20260925 09:20:14.156044 7674 raft_consensus.cc:2454] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5bf437e4e24e9ba208017a4bb49b23 in term 0.
I20260925 09:20:14.156189 7807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e41200ecc46e9b41ba65b934c5da3" candidate_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e60e94b5979470a99c9510be38bd497" is_pre_election: true
I20260925 09:20:14.156692 7807 raft_consensus.cc:2454] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5bf437e4e24e9ba208017a4bb49b23 in term 0.
I20260925 09:20:14.156836 7873 leader_election.cc:302] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190220a8090a4b7aa6d66bf0685f3151, 4a5bf437e4e24e9ba208017a4bb49b23; no voters:
I20260925 09:20:14.157331 8012 raft_consensus.cc:2781] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:14.157502 8012 raft_consensus.cc:482] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:14.157610 8012 raft_consensus.cc:3037] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:14.160108 8012 raft_consensus.cc:504] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.160852 8012 leader_election.cc:288] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [CANDIDATE]: Term 1 election: Requested vote from peers 190220a8090a4b7aa6d66bf0685f3151 (127.3.82.1:40525), 0e60e94b5979470a99c9510be38bd497 (127.3.82.2:34027)
I20260925 09:20:14.161535 7674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e41200ecc46e9b41ba65b934c5da3" candidate_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "190220a8090a4b7aa6d66bf0685f3151"
I20260925 09:20:14.161785 7674 raft_consensus.cc:3037] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:14.161819 7807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e41200ecc46e9b41ba65b934c5da3" candidate_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e60e94b5979470a99c9510be38bd497"
I20260925 09:20:14.162122 7807 raft_consensus.cc:3037] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:14.164613 7807 raft_consensus.cc:2454] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5bf437e4e24e9ba208017a4bb49b23 in term 1.
I20260925 09:20:14.164894 7674 raft_consensus.cc:2454] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5bf437e4e24e9ba208017a4bb49b23 in term 1.
I20260925 09:20:14.165201 7875 leader_election.cc:302] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e60e94b5979470a99c9510be38bd497, 4a5bf437e4e24e9ba208017a4bb49b23; no voters:
I20260925 09:20:14.165696 8012 raft_consensus.cc:2781] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:14.166227 8012 raft_consensus.cc:686] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 1 LEADER]: Becoming Leader. State: Replica: 4a5bf437e4e24e9ba208017a4bb49b23, State: Running, Role: LEADER
I20260925 09:20:14.166787 8012 consensus_queue.cc:237] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } }
I20260925 09:20:14.172338 7531 catalog_manager.cc:5495] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a5bf437e4e24e9ba208017a4bb49b23 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a5bf437e4e24e9ba208017a4bb49b23" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46123 } health_report { overall_health: HEALTHY } } }
W20260925 09:20:14.189885 7853 tablet.cc:2369] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:20:14.217641 7986 tablet.cc:2369] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:14.219106 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.212137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.3.82.3:46123)
0925 09:20:14.212450 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.212456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.212476 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:20:14.212695 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.212699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.212777 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.212912 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:14.213334 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.214207 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.216008 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.216488 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.216587 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.217256 (+ 669us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:14.218074 (+ 818us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:14.218224 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.218742 (+ 518us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.218836 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260925 09:20:14.229652 7807 raft_consensus.cc:1261] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 1 FOLLOWER]: Refusing update from remote peer 4a5bf437e4e24e9ba208017a4bb49b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:14.229872 7674 raft_consensus.cc:1261] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 1 FOLLOWER]: Refusing update from remote peer 4a5bf437e4e24e9ba208017a4bb49b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:14.230836 8012 consensus_queue.cc:1035] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e60e94b5979470a99c9510be38bd497" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 34027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:14.231179 8014 consensus_queue.cc:1035] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190220a8090a4b7aa6d66bf0685f3151" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:14.243701 8018 mvcc.cc:205] Tried to move back new op lower bound from 7333183546273763328 to 7333183546031738880. Current Snapshot: MvccSnapshot[applied={T|T < 7333183546273763328}]
I20260925 09:20:14.261953 7531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:14.262562 7531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:14.266963 7531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0969db43c87e4df69c03aeddd68bce3b: Sending DeleteTablet for 3 replicas of tablet 759e41200ecc46e9b41ba65b934c5da3
I20260925 09:20:14.268232 7654 tablet_service.cc:1514] Processing DeleteTablet for tablet 759e41200ecc46e9b41ba65b934c5da3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:14 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:57904
I20260925 09:20:14.268288 7787 tablet_service.cc:1514] Processing DeleteTablet for tablet 759e41200ecc46e9b41ba65b934c5da3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:14 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:38026
I20260925 09:20:14.269273 8026 tablet_replica.cc:330] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: stopping tablet replica
I20260925 09:20:14.269953 8026 raft_consensus.cc:2229] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:14.269970 7920 tablet_service.cc:1514] Processing DeleteTablet for tablet 759e41200ecc46e9b41ba65b934c5da3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:14 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:54846
I20260925 09:20:14.270707 8026 raft_consensus.cc:2258] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:14.271476 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7592
I20260925 09:20:14.273518 8026 ts_tablet_manager.cc:1905] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:14.280718 8027 tablet_replica.cc:330] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: stopping tablet replica
I20260925 09:20:14.281258 8027 raft_consensus.cc:2229] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:14.282065 8027 raft_consensus.cc:2258] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:14.283181 8026 ts_tablet_manager.cc:1918] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:14.283465 8026 log.cc:1196] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/759e41200ecc46e9b41ba65b934c5da3
I20260925 09:20:14.284139 8026 ts_tablet_manager.cc:1939] T 759e41200ecc46e9b41ba65b934c5da3 P 190220a8090a4b7aa6d66bf0685f3151: Deleting consensus metadata
I20260925 09:20:14.284546 8027 ts_tablet_manager.cc:1905] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:14.286537 7515 catalog_manager.cc:4841] TS 190220a8090a4b7aa6d66bf0685f3151 (127.3.82.1:40525): tablet 759e41200ecc46e9b41ba65b934c5da3 (table test-table [id=90eb6a14c92c4147a986dee2f81b0d63]) successfully deleted
I20260925 09:20:14.291213 8028 tablet_replica.cc:330] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: stopping tablet replica
I20260925 09:20:14.291848 8028 raft_consensus.cc:2229] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:14.291951 8027 ts_tablet_manager.cc:1918] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:14.292138 8027 log.cc:1196] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/759e41200ecc46e9b41ba65b934c5da3
I20260925 09:20:14.292752 8027 ts_tablet_manager.cc:1939] T 759e41200ecc46e9b41ba65b934c5da3 P 0e60e94b5979470a99c9510be38bd497: Deleting consensus metadata
I20260925 09:20:14.292795 8028 raft_consensus.cc:2258] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:14.294785 7517 catalog_manager.cc:4841] TS 0e60e94b5979470a99c9510be38bd497 (127.3.82.2:34027): tablet 759e41200ecc46e9b41ba65b934c5da3 (table test-table [id=90eb6a14c92c4147a986dee2f81b0d63]) successfully deleted
I20260925 09:20:14.295557 8028 ts_tablet_manager.cc:1905] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 09:20:14 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:14.302471 8028 ts_tablet_manager.cc:1918] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:14.302670 8028 log.cc:1196] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/759e41200ecc46e9b41ba65b934c5da3
I20260925 09:20:14.303202 8028 ts_tablet_manager.cc:1939] T 759e41200ecc46e9b41ba65b934c5da3 P 4a5bf437e4e24e9ba208017a4bb49b23: Deleting consensus metadata
I20260925 09:20:14.305290 7518 catalog_manager.cc:4841] TS 4a5bf437e4e24e9ba208017a4bb49b23 (127.3.82.3:46123): tablet 759e41200ecc46e9b41ba65b934c5da3 (table test-table [id=90eb6a14c92c4147a986dee2f81b0d63]) successfully deleted
I20260925 09:20:14.309554 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.272776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.3.82.1:40525)
0925 09:20:14.272962 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.272968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.290454 (+ 17486us) server_negotiation.cc:407] Connection header received
0925 09:20:14.290683 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.290692 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.290816 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.290937 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:14.292798 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.293972 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.295843 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.296508 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.296651 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.302128 (+ 5477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:14.302150 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:14.302172 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:14.302234 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:14.306206 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.307230 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.307238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.307241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.307396 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.307854 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.307857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.307859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.308325 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:14.308628 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.308966 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.309199 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:20:14.310387 7714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:14.401978 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7592
I20260925 09:20:14.426088 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7725
Sep 25 09:20:14 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:20:14.446650 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.427051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.3.82.2:34027)
0925 09:20:14.427223 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.427227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.427246 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:20:14.427354 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.427358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.427441 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.427522 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:14.429116 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.430056 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.431803 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.432201 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.432297 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.438301 (+ 6004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:14.438322 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:14.438344 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:14.438410 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:14.442862 (+ 4452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.443980 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.443991 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.443995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.444189 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.444682 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.444688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.444692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.445245 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:14.445568 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.445966 (+ 398us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.446254 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:20:14.447476 7847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:14.561573 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7725
I20260925 09:20:14.579654 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7858
Sep 25 09:20:14 dist-test-slave-4wm5 krb5kdc[7487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:20:14.597796 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:14.580579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54862 (local address 127.3.82.3:46123)
0925 09:20:14.580781 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:14.580787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:14.580812 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:20:14.580967 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:14.580976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:14.581066 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:14.581202 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:14.582676 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.583550 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.584854 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:14.585340 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:14.585485 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:14.590709 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:14.590727 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:14.590744 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:14.590797 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:14.594558 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.595410 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.595418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.595420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.595604 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:14.596055 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:14.596060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:14.596068 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:14.596454 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:14.596825 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:14.597139 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 09:20:14.597405 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:20:14.598692 7980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:14.726104 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7858
I20260925 09:20:14.746448 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 7500
I20260925 09:20:14.747792 7561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:14.859525 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 7500
2026-09-25T09:20:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4476 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T09:20:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:14Z Disabled control of system clock
I20260925 09:20:14.930222 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45425
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:45735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 09:20:15.283754 8037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:15.284198 8037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:15.284363 8037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:15.291980 8037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:15.292150 8037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:15.292239 8037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:15.292343 8037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:15.292428 8037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:15.302106 8037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45425
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:15.303519 8037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:15.305397 8037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:15.318109 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:15.318122 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:15.318147 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:15.319022 8037 server_base.cc:1034] running on GCE node
I20260925 09:20:15.319840 8037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:15.321506 8037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:15.322777 8037 hybrid_clock.cc:648] HybridClock initialized: now 1790328015322696 us; error 78 us; skew 500 ppm
I20260925 09:20:15.323197 8037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:15.325680 8037 webserver.cc:460] Webserver started at http://127.3.82.62:40143/ using document root <none> and password file <none>
I20260925 09:20:15.326511 8037 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:15.326635 8037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:15.327018 8037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:15.330348 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "18249e71e3444071bd260299cdedc7bb"
format_stamp: "Formatted at 2026-09-25 09:20:15 on dist-test-slave-4wm5"
server_key: "b1c700843d49273392ecba94b9cc6c0b"
server_key_iv: "5749c44e3b9f60b6d8eb59585c4ec03e"
server_key_version: "encryptionkey@0"
I20260925 09:20:15.331282 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "18249e71e3444071bd260299cdedc7bb"
format_stamp: "Formatted at 2026-09-25 09:20:15 on dist-test-slave-4wm5"
server_key: "b1c700843d49273392ecba94b9cc6c0b"
server_key_iv: "5749c44e3b9f60b6d8eb59585c4ec03e"
server_key_version: "encryptionkey@0"
I20260925 09:20:15.337037 8037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:20:15.341105 8051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:15.342777 8037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:15.342989 8037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "18249e71e3444071bd260299cdedc7bb"
format_stamp: "Formatted at 2026-09-25 09:20:15 on dist-test-slave-4wm5"
server_key: "b1c700843d49273392ecba94b9cc6c0b"
server_key_iv: "5749c44e3b9f60b6d8eb59585c4ec03e"
server_key_version: "encryptionkey@0"
I20260925 09:20:15.343236 8037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:15.349396 8037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:15.350391 8037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:15.350713 8037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:15.370379 8037 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45425
I20260925 09:20:15.370402 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45425 every 8 connection(s)
I20260925 09:20:15.371883 8037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:15.375950 8103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:15.381848 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8037
I20260925 09:20:15.382063 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:15.382871 3400 external_mini_cluster.cc:1428] Setting key 9bed2aae17630d19b8c690be93e64621
I20260925 09:20:15.388192 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb: Bootstrap starting.
I20260925 09:20:15.391795 8103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:15.392158 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:15.385246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.3.82.62:45425)
0925 09:20:15.386419 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:15.386431 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:15.386474 (+ 43us) server_negotiation.cc:407] Connection header received
0925 09:20:15.388181 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:15.388208 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:15.388797 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:15.389206 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:15.390410 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:15.390417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:15.390430 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:15.390445 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:15.390509 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:15.390525 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:15.390766 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:15.390916 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 09:20:15.391170 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":359,"threads_started":1}
I20260925 09:20:15.393234 8103 log.cc:824] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:15.396395 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb: No bootstrap required, opened a new log
I20260925 09:20:15.404018 8103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18249e71e3444071bd260299cdedc7bb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45425 } }
I20260925 09:20:15.404414 8103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:15.404495 8103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18249e71e3444071bd260299cdedc7bb, State: Initialized, Role: FOLLOWER
I20260925 09:20:15.405109 8103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18249e71e3444071bd260299cdedc7bb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45425 } }
I20260925 09:20:15.405373 8103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:15.405529 8103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:15.405725 8103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:15.408146 8103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18249e71e3444071bd260299cdedc7bb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45425 } }
I20260925 09:20:15.408671 8103 leader_election.cc:302] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18249e71e3444071bd260299cdedc7bb; no voters:
I20260925 09:20:15.409309 8103 leader_election.cc:288] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:15.409715 8108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:15.410701 8108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [term 1 LEADER]: Becoming Leader. State: Replica: 18249e71e3444071bd260299cdedc7bb, State: Running, Role: LEADER
I20260925 09:20:15.411523 8108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18249e71e3444071bd260299cdedc7bb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45425 } }
I20260925 09:20:15.412166 8103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:15.414682 8109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18249e71e3444071bd260299cdedc7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18249e71e3444071bd260299cdedc7bb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45425 } } }
I20260925 09:20:15.415102 8109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:15.415130 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18249e71e3444071bd260299cdedc7bb. Latest consensus state: current_term: 1 leader_uuid: "18249e71e3444071bd260299cdedc7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18249e71e3444071bd260299cdedc7bb" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45425 } } }
I20260925 09:20:15.415529 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:15.416389 8117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:15.422363 8117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:15.431998 8117 catalog_manager.cc:1347] Generated new cluster ID: d6285a91617e4453b5d8b3b9a5a0f3eb
I20260925 09:20:15.432129 8117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:15.461045 8117 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:15.461664 8117 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:15.472432 8117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb: Generated new TSK 0
I20260925 09:20:15.473495 8117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:20:15.483266 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45425
--builtin_ntp_servers=127.3.82.20:45735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 09:20:15.837653 8127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:15.838053 8127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:15.838230 8127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:15.846221 8127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:15.846350 8127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:15.846573 8127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:15.856505 8127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:15.858007 8127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:15.859968 8127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:15.875078 8133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:15.875064 8135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:15.875411 8132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:15.876123 8127 server_base.cc:1034] running on GCE node
I20260925 09:20:15.876830 8127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:15.877960 8127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:15.879165 8127 hybrid_clock.cc:648] HybridClock initialized: now 1790328015879087 us; error 74 us; skew 500 ppm
I20260925 09:20:15.879519 8127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:15.882102 8127 webserver.cc:460] Webserver started at http://127.3.82.1:41585/ using document root <none> and password file <none>
I20260925 09:20:15.882884 8127 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:15.882977 8127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:15.883333 8127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:15.886705 8127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "39824e91c45444df899361792be7411b"
format_stamp: "Formatted at 2026-09-25 09:20:15 on dist-test-slave-4wm5"
server_key: "e483bd034aa4205582f452a812e64f3b"
server_key_iv: "154f255682c73413a2087b9da5a0a5fd"
server_key_version: "encryptionkey@0"
I20260925 09:20:15.887594 8127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "39824e91c45444df899361792be7411b"
format_stamp: "Formatted at 2026-09-25 09:20:15 on dist-test-slave-4wm5"
server_key: "e483bd034aa4205582f452a812e64f3b"
server_key_iv: "154f255682c73413a2087b9da5a0a5fd"
server_key_version: "encryptionkey@0"
I20260925 09:20:15.893150 8127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:20:15.897220 8141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:15.898795 8127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:20:15.899008 8127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "39824e91c45444df899361792be7411b"
format_stamp: "Formatted at 2026-09-25 09:20:15 on dist-test-slave-4wm5"
server_key: "e483bd034aa4205582f452a812e64f3b"
server_key_iv: "154f255682c73413a2087b9da5a0a5fd"
server_key_version: "encryptionkey@0"
I20260925 09:20:15.899268 8127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:15.905541 8127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:15.906530 8127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:15.906823 8127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:15.907819 8127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:15.909369 8127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:15.909502 8127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:15.909637 8127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:15.909734 8127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:15.945619 8127 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:46063
I20260925 09:20:15.945658 8253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:46063 every 8 connection(s)
I20260925 09:20:15.947218 8127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:15.956130 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:15.949958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:54497 (local address 127.3.82.62:45425)
0925 09:20:15.950428 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:15.950434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:15.951826 (+ 1392us) server_negotiation.cc:407] Connection header received
0925 09:20:15.953574 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:15.953579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:15.953640 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:15.953767 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:15.955371 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:15.955376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:15.955379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:15.955395 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:15.955463 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:15.955476 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:15.955582 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:15.955851 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 09:20:15.955907 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":234,"threads_started":1}
I20260925 09:20:15.956174 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8127
I20260925 09:20:15.956377 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:15.957191 3400 external_mini_cluster.cc:1428] Setting key cea99729608e0a7fa8de788238cc6511
I20260925 09:20:15.957641 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:15.950587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45425 (local address 127.3.82.1:54497)
0925 09:20:15.951658 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:15.951695 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:15.953279 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:15.954149 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:15.954158 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:15.954274 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:15.955031 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:15.955033 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:15.955086 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:15.955184 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:15.955569 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:15.955576 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:15.955582 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 09:20:15.955964 (+ 382us) client_negotiation.cc:240] Negotiation successful
0925 09:20:15.956282 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":383,"threads_started":1}
I20260925 09:20:15.959365 8254 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45425
I20260925 09:20:15.959738 8254 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:15.960197 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45425
--builtin_ntp_servers=127.3.82.20:45735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 09:20:15.960479 8254 heartbeater.cc:511] Master 127.3.82.62:45425 requested a full tablet report, sending...
I20260925 09:20:15.962942 8068 ts_manager.cc:194] Registered new tserver with Master: 39824e91c45444df899361792be7411b (127.3.82.1:46063)
W20260925 09:20:16.308396 8258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:16.308715 8258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:16.308885 8258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:16.317728 8258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:16.317921 8258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:16.318151 8258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:16.328578 8258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:16.330101 8258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:16.332137 8258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:16.346638 8263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:16.346822 8264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:16.347054 8266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:16.347671 8258 server_base.cc:1034] running on GCE node
I20260925 09:20:16.348387 8258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:16.349541 8258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:16.350764 8258 hybrid_clock.cc:648] HybridClock initialized: now 1790328016350685 us; error 73 us; skew 500 ppm
I20260925 09:20:16.351083 8258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:16.353780 8258 webserver.cc:460] Webserver started at http://127.3.82.2:34403/ using document root <none> and password file <none>
I20260925 09:20:16.354682 8258 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:16.354811 8258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:16.355167 8258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:16.358373 8258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "4fb3f29e4816415abcba39cd21e42e76"
format_stamp: "Formatted at 2026-09-25 09:20:16 on dist-test-slave-4wm5"
server_key: "74fb2ef7444e0eb2baeee810512e73a1"
server_key_iv: "e4fd4170e164c6ffabc7420bf05a0833"
server_key_version: "encryptionkey@0"
I20260925 09:20:16.359227 8258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "4fb3f29e4816415abcba39cd21e42e76"
format_stamp: "Formatted at 2026-09-25 09:20:16 on dist-test-slave-4wm5"
server_key: "74fb2ef7444e0eb2baeee810512e73a1"
server_key_iv: "e4fd4170e164c6ffabc7420bf05a0833"
server_key_version: "encryptionkey@0"
I20260925 09:20:16.364976 8258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:20:16.368866 8272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:16.370520 8258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:16.370707 8258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "4fb3f29e4816415abcba39cd21e42e76"
format_stamp: "Formatted at 2026-09-25 09:20:16 on dist-test-slave-4wm5"
server_key: "74fb2ef7444e0eb2baeee810512e73a1"
server_key_iv: "e4fd4170e164c6ffabc7420bf05a0833"
server_key_version: "encryptionkey@0"
I20260925 09:20:16.370962 8258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:16.378014 8258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:16.379177 8258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:16.379523 8258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:16.380918 8258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:16.383077 8258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:16.383168 8258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:16.383301 8258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:16.383410 8258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:16.418699 8258 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:46521
I20260925 09:20:16.418777 8384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:46521 every 8 connection(s)
I20260925 09:20:16.420099 8258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:16.422890 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8258
I20260925 09:20:16.423101 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:16.423704 3400 external_mini_cluster.cc:1428] Setting key 5ed104dd6e64249890c4c23a7b04598b
I20260925 09:20:16.426640 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45425
--builtin_ntp_servers=127.3.82.20:45735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 09:20:16.429340 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.423118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:40659 (local address 127.3.82.62:45425)
0925 09:20:16.424225 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:16.424233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:16.425018 (+ 785us) server_negotiation.cc:407] Connection header received
0925 09:20:16.426895 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:16.426901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:16.426973 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:16.427103 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:16.428581 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:16.428587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:16.428592 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:16.428619 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:16.428652 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:16.428664 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:16.428807 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:16.428997 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 09:20:16.429074 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935}
I20260925 09:20:16.430481 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.423694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45425 (local address 127.3.82.2:40659)
0925 09:20:16.424844 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:16.424897 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:16.426594 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:16.427371 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:16.427380 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:16.427491 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:16.428238 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:16.428240 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:16.428281 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:16.428383 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:16.428762 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:16.428773 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:16.428784 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:20:16.429092 (+ 308us) client_negotiation.cc:240] Negotiation successful
0925 09:20:16.429404 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":473,"threads_started":1}
I20260925 09:20:16.432120 8385 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45425
I20260925 09:20:16.432518 8385 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:16.433279 8385 heartbeater.cc:511] Master 127.3.82.62:45425 requested a full tablet report, sending...
I20260925 09:20:16.435268 8068 ts_manager.cc:194] Registered new tserver with Master: 4fb3f29e4816415abcba39cd21e42e76 (127.3.82.2:46521)
W20260925 09:20:16.779688 8388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:16.780009 8388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:16.780108 8388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:16.787719 8388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:16.787854 8388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:16.788036 8388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:16.797708 8388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:16.799109 8388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:16.801064 8388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:16.815802 8394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:16.815907 8393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:16.816116 8396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:16.816926 8388 server_base.cc:1034] running on GCE node
I20260925 09:20:16.817549 8388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:16.818671 8388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:16.819931 8388 hybrid_clock.cc:648] HybridClock initialized: now 1790328016819871 us; error 66 us; skew 500 ppm
I20260925 09:20:16.820284 8388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:16.822959 8388 webserver.cc:460] Webserver started at http://127.3.82.3:36995/ using document root <none> and password file <none>
I20260925 09:20:16.823755 8388 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:16.823905 8388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:16.824244 8388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:16.827555 8388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "90f1571ada7b443491c960ffc65f6954"
format_stamp: "Formatted at 2026-09-25 09:20:16 on dist-test-slave-4wm5"
server_key: "b1594f29cd5636f1fbe11c5b42d4106a"
server_key_iv: "e5eeca5394c71996caef2be9dadfda1c"
server_key_version: "encryptionkey@0"
I20260925 09:20:16.828501 8388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "90f1571ada7b443491c960ffc65f6954"
format_stamp: "Formatted at 2026-09-25 09:20:16 on dist-test-slave-4wm5"
server_key: "b1594f29cd5636f1fbe11c5b42d4106a"
server_key_iv: "e5eeca5394c71996caef2be9dadfda1c"
server_key_version: "encryptionkey@0"
I20260925 09:20:16.834416 8388 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:20:16.838682 8402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:16.840520 8388 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:20:16.840741 8388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "90f1571ada7b443491c960ffc65f6954"
format_stamp: "Formatted at 2026-09-25 09:20:16 on dist-test-slave-4wm5"
server_key: "b1594f29cd5636f1fbe11c5b42d4106a"
server_key_iv: "e5eeca5394c71996caef2be9dadfda1c"
server_key_version: "encryptionkey@0"
I20260925 09:20:16.841013 8388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:16.847258 8388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:16.848167 8388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:16.848465 8388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:16.849483 8388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:16.851039 8388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:16.851166 8388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:16.851312 8388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:16.851403 8388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:16.887877 8388 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:42545
I20260925 09:20:16.887980 8514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:42545 every 8 connection(s)
I20260925 09:20:16.889372 8388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:16.890761 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8388
I20260925 09:20:16.890926 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:16.891527 3400 external_mini_cluster.cc:1428] Setting key 9b736503e77c1cdbd1cb367168fe3a40
I20260925 09:20:16.899223 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.892635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50979 (local address 127.3.82.62:45425)
0925 09:20:16.892838 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:16.892847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:16.894792 (+ 1945us) server_negotiation.cc:407] Connection header received
0925 09:20:16.896625 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:16.896630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:16.896700 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:16.896827 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:16.898406 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:16.898412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:16.898417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:16.898443 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:16.898476 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:16.898488 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:16.898630 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:16.898857 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 09:20:16.898933 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:20:16.900326 8517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.893308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45425 (local address 127.3.82.3:50979)
0925 09:20:16.894613 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:16.894674 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:16.896341 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:16.897085 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:16.897093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:16.897196 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:16.897991 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:16.897994 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:16.898044 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:16.898140 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:16.898590 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:16.898602 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:16.898612 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:20:16.898952 (+ 340us) client_negotiation.cc:240] Negotiation successful
0925 09:20:16.899285 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":399,"threads_started":1}
I20260925 09:20:16.902221 8515 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45425
I20260925 09:20:16.902678 8515 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:16.903427 8515 heartbeater.cc:511] Master 127.3.82.62:45425 requested a full tablet report, sending...
I20260925 09:20:16.905189 8068 ts_manager.cc:194] Registered new tserver with Master: 90f1571ada7b443491c960ffc65f6954 (127.3.82.3:42545)
I20260925 09:20:16.907671 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:20:16.915367 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.912632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.3.82.62:45425)
0925 09:20:16.912815 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:16.912820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:16.913233 (+ 413us) server_negotiation.cc:407] Connection header received
0925 09:20:16.913421 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:16.913427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:16.913495 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:16.913618 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:16.914664 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:16.914671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:16.914675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:16.914695 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:16.914736 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:16.914749 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:16.914912 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:16.914999 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 09:20:16.915065 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 09:20:16.919790 8068 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:16.926868 8068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:16.945601 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.940202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:46521 (local address 127.0.0.1:59664)
0925 09:20:16.941044 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:16.941100 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:16.941228 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:16.942436 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:16.942449 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:16.942553 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:16.943463 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:16.943466 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:16.943498 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:16.943592 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:16.944850 (+ 1258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:16.944862 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:16.944869 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:20:16.945058 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 09:20:16.945176 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":363,"threads_started":1}
I20260925 09:20:16.946343 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.940435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.3.82.2:46521)
0925 09:20:16.941732 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:16.941741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:16.941788 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:20:16.941897 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:16.941905 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:16.942197 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:16.943120 (+ 923us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:16.944623 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:16.944633 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:16.944654 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:16.944682 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:16.944742 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:16.944767 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:16.944896 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:16.945896 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0925 09:20:16.946033 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":979,"threads_started":1}
I20260925 09:20:16.950181 8531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.942608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.3.82.3:42545)
0925 09:20:16.945534 (+ 2926us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:16.945542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:16.945581 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:20:16.945649 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:16.945657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:16.946230 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:16.946401 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:16.948566 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:16.948577 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:16.948593 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:16.948616 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:16.948686 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:16.948713 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:16.948877 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:16.949674 (+ 797us) server_negotiation.cc:299] Negotiation successful
0925 09:20:16.949789 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2734,"thread_start_us":2579,"threads_started":1}
I20260925 09:20:16.950659 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.942364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:42545 (local address 127.0.0.1:39222)
0925 09:20:16.944608 (+ 2244us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:16.944628 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:16.944727 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:16.946805 (+ 2078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:16.946811 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:16.946841 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:16.948304 (+ 1463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:16.948308 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:16.948328 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:16.948375 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:16.948966 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:16.948970 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:16.948975 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:20:16.949102 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 09:20:16.949156 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":695,"threads_started":1}
I20260925 09:20:16.952697 8320 tablet_service.cc:1467] Processing CreateTablet for tablet 435e97b527ae477b8dbe318ff1dbd135 (DEFAULT_TABLE table=test-table [id=c5f00fb51511440d9f64c53070e50250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:16.955283 8320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435e97b527ae477b8dbe318ff1dbd135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:16.957015 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.941554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60420)
0925 09:20:16.942428 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:16.942450 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:16.942532 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:16.954133 (+ 11601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:16.954138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:16.954172 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:16.954934 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:16.954938 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:16.954957 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:16.955009 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:16.956303 (+ 1294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:16.956309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:16.956315 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 09:20:16.956479 (+ 164us) client_negotiation.cc:240] Negotiation successful
0925 09:20:16.956543 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":538,"threads_started":1}
I20260925 09:20:16.956471 8450 tablet_service.cc:1467] Processing CreateTablet for tablet 435e97b527ae477b8dbe318ff1dbd135 (DEFAULT_TABLE table=test-table [id=c5f00fb51511440d9f64c53070e50250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:16.957993 8450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435e97b527ae477b8dbe318ff1dbd135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:16.958039 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:16.942828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.3.82.1:46063)
0925 09:20:16.953526 (+ 10698us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:16.953534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:16.953565 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:20:16.953638 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:16.953646 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:16.953908 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:16.954637 (+ 729us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:16.956047 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:16.956055 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:16.956069 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:16.956087 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:16.956145 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:16.956164 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:16.956340 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:16.956473 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 09:20:16.957632 (+ 1159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10500,"thread_start_us":10292,"threads_started":1}
I20260925 09:20:16.962222 8189 tablet_service.cc:1467] Processing CreateTablet for tablet 435e97b527ae477b8dbe318ff1dbd135 (DEFAULT_TABLE table=test-table [id=c5f00fb51511440d9f64c53070e50250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:16.964438 8189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435e97b527ae477b8dbe318ff1dbd135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:16.966605 8254 heartbeater.cc:503] Master 127.3.82.62:45425 was elected leader, sending a full tablet report...
I20260925 09:20:16.968518 8533 tablet_bootstrap.cc:492] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Bootstrap starting.
I20260925 09:20:16.972251 8533 tablet_bootstrap.cc:654] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:16.973201 8534 tablet_bootstrap.cc:492] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Bootstrap starting.
I20260925 09:20:16.973660 8533 log.cc:824] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:16.977351 8533 tablet_bootstrap.cc:492] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: No bootstrap required, opened a new log
I20260925 09:20:16.977321 8534 tablet_bootstrap.cc:654] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:16.977648 8533 ts_tablet_manager.cc:1397] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260925 09:20:16.978698 8534 log.cc:824] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:16.981977 8537 tablet_bootstrap.cc:492] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: Bootstrap starting.
I20260925 09:20:16.983146 8534 tablet_bootstrap.cc:492] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: No bootstrap required, opened a new log
I20260925 09:20:16.983453 8534 ts_tablet_manager.cc:1397] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260925 09:20:16.985361 8537 tablet_bootstrap.cc:654] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:16.986559 8537 log.cc:824] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:16.989722 8537 tablet_bootstrap.cc:492] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: No bootstrap required, opened a new log
I20260925 09:20:16.989464 8533 raft_consensus.cc:348] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:16.990013 8533 raft_consensus.cc:374] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:16.990160 8537 ts_tablet_manager.cc:1397] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260925 09:20:16.990159 8533 raft_consensus.cc:729] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb3f29e4816415abcba39cd21e42e76, State: Initialized, Role: FOLLOWER
I20260925 09:20:16.990856 8533 consensus_queue.cc:260] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:16.995901 8534 raft_consensus.cc:348] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:16.996701 8534 raft_consensus.cc:374] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:16.996896 8534 raft_consensus.cc:729] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f1571ada7b443491c960ffc65f6954, State: Initialized, Role: FOLLOWER
I20260925 09:20:16.998063 8534 consensus_queue.cc:260] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:17.000965 8515 heartbeater.cc:503] Master 127.3.82.62:45425 was elected leader, sending a full tablet report...
I20260925 09:20:17.001094 8537 raft_consensus.cc:348] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:17.001590 8537 raft_consensus.cc:374] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:17.001749 8537 raft_consensus.cc:729] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39824e91c45444df899361792be7411b, State: Initialized, Role: FOLLOWER
I20260925 09:20:17.002023 8534 ts_tablet_manager.cc:1428] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260925 09:20:17.002651 8537 consensus_queue.cc:260] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:17.004158 8533 ts_tablet_manager.cc:1428] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Time spent starting tablet: real 0.026s user 0.020s sys 0.000s
I20260925 09:20:17.004619 8385 heartbeater.cc:503] Master 127.3.82.62:45425 was elected leader, sending a full tablet report...
I20260925 09:20:17.005100 8537 ts_tablet_manager.cc:1428] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260925 09:20:17.141235 8516 tablet.cc:2369] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:20:17.172339 8386 tablet.cc:2369] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:20:17.199816 8255 tablet.cc:2369] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:17.232532 8541 raft_consensus.cc:482] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:17.232874 8541 raft_consensus.cc:504] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:17.234797 8541 leader_election.cc:288] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fb3f29e4816415abcba39cd21e42e76 (127.3.82.2:46521), 90f1571ada7b443491c960ffc65f6954 (127.3.82.3:42545)
I20260925 09:20:17.239959 8531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.235463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:36825 (local address 127.3.82.3:42545)
0925 09:20:17.235663 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:17.235670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:17.237510 (+ 1840us) server_negotiation.cc:407] Connection header received
0925 09:20:17.237724 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:17.237730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:17.237804 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:17.237916 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:17.239169 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:17.239175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:17.239179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:17.239200 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:17.239239 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:17.239253 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:17.239353 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:17.239596 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 09:20:17.239667 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:20:17.239959 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.235157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:45001 (local address 127.3.82.2:46521)
0925 09:20:17.235372 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:17.235381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:17.237037 (+ 1656us) server_negotiation.cc:407] Connection header received
0925 09:20:17.237511 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:17.237517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:17.237591 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:17.237695 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:17.238895 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:17.238904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:17.238908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:17.238940 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:17.238981 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:17.238999 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:17.239143 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:17.239642 (+ 499us) server_negotiation.cc:299] Negotiation successful
0925 09:20:17.239702 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 09:20:17.240183 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.235175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:46521 (local address 127.3.82.1:45001)
0925 09:20:17.236848 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.236876 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:17.237236 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:17.237752 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:17.237756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:17.237787 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:17.238629 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:17.238633 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:17.238651 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:17.238735 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:17.239505 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:17.239510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:17.239515 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:20:17.239673 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 09:20:17.239735 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"thread_start_us":1795,"threads_started":1}
I20260925 09:20:17.240752 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.235354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:42545 (local address 127.3.82.1:36825)
0925 09:20:17.237390 (+ 2036us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.237416 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:17.237545 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:17.238126 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:17.238132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:17.238160 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:17.238912 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:20:17.238915 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:17.238929 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:20:17.238991 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:17.239445 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:17.239450 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:17.239454 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 09:20:17.240228 (+ 774us) client_negotiation.cc:240] Negotiation successful
0925 09:20:17.240287 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1820,"thread_start_us":827,"threads_started":1}
I20260925 09:20:17.241310 8340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435e97b527ae477b8dbe318ff1dbd135" candidate_uuid: "39824e91c45444df899361792be7411b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb3f29e4816415abcba39cd21e42e76" is_pre_election: true
I20260925 09:20:17.241770 8470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435e97b527ae477b8dbe318ff1dbd135" candidate_uuid: "39824e91c45444df899361792be7411b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1571ada7b443491c960ffc65f6954" is_pre_election: true
I20260925 09:20:17.241983 8340 raft_consensus.cc:2454] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39824e91c45444df899361792be7411b in term 0.
I20260925 09:20:17.242327 8470 raft_consensus.cc:2454] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39824e91c45444df899361792be7411b in term 0.
I20260925 09:20:17.242830 8142 leader_election.cc:302] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39824e91c45444df899361792be7411b, 4fb3f29e4816415abcba39cd21e42e76; no voters:
I20260925 09:20:17.243268 8541 raft_consensus.cc:2781] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:17.243412 8541 raft_consensus.cc:482] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:17.243515 8541 raft_consensus.cc:3037] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:17.245942 8541 raft_consensus.cc:504] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:17.246738 8541 leader_election.cc:288] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [CANDIDATE]: Term 1 election: Requested vote from peers 4fb3f29e4816415abcba39cd21e42e76 (127.3.82.2:46521), 90f1571ada7b443491c960ffc65f6954 (127.3.82.3:42545)
I20260925 09:20:17.247284 8470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435e97b527ae477b8dbe318ff1dbd135" candidate_uuid: "39824e91c45444df899361792be7411b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f1571ada7b443491c960ffc65f6954"
I20260925 09:20:17.247357 8340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435e97b527ae477b8dbe318ff1dbd135" candidate_uuid: "39824e91c45444df899361792be7411b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb3f29e4816415abcba39cd21e42e76"
I20260925 09:20:17.247509 8470 raft_consensus.cc:3037] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:17.247627 8340 raft_consensus.cc:3037] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:17.250080 8470 raft_consensus.cc:2454] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39824e91c45444df899361792be7411b in term 1.
I20260925 09:20:17.250080 8340 raft_consensus.cc:2454] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39824e91c45444df899361792be7411b in term 1.
I20260925 09:20:17.250573 8142 leader_election.cc:302] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39824e91c45444df899361792be7411b, 4fb3f29e4816415abcba39cd21e42e76; no voters:
I20260925 09:20:17.250929 8541 raft_consensus.cc:2781] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:17.251415 8541 raft_consensus.cc:686] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [term 1 LEADER]: Becoming Leader. State: Replica: 39824e91c45444df899361792be7411b, State: Running, Role: LEADER
I20260925 09:20:17.251981 8541 consensus_queue.cc:237] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } }
I20260925 09:20:17.257042 8066 catalog_manager.cc:5495] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b reported cstate change: term changed from 0 to 1, leader changed from <none> to 39824e91c45444df899361792be7411b (127.3.82.1). New cstate: current_term: 1 leader_uuid: "39824e91c45444df899361792be7411b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39824e91c45444df899361792be7411b" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 } health_report { overall_health: UNKNOWN } } }
I20260925 09:20:17.323871 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.321497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60428 (local address 127.3.82.1:46063)
0925 09:20:17.321674 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:17.321679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:17.321697 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:20:17.321989 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:17.321993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:17.322078 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:17.322168 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:17.323203 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:17.323207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:17.323210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:17.323226 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:17.323257 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:17.323269 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:17.323377 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:17.323527 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 09:20:17.323575 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:20:17.332958 8340 raft_consensus.cc:1261] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 1 FOLLOWER]: Refusing update from remote peer 39824e91c45444df899361792be7411b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:17.332978 8470 raft_consensus.cc:1261] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 1 FOLLOWER]: Refusing update from remote peer 39824e91c45444df899361792be7411b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:17.334273 8544 consensus_queue.cc:1035] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f1571ada7b443491c960ffc65f6954" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:17.334770 8541 consensus_queue.cc:1035] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fb3f29e4816415abcba39cd21e42e76" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 46521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:17.347714 8547 mvcc.cc:205] Tried to move back new op lower bound from 7333183558984396800 to 7333183558667763712. Current Snapshot: MvccSnapshot[applied={T|T < 7333183558984396800}]
I20260925 09:20:17.364576 8068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:17.364835 8068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:17.368882 8068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18249e71e3444071bd260299cdedc7bb: Sending DeleteTablet for 3 replicas of tablet 435e97b527ae477b8dbe318ff1dbd135
I20260925 09:20:17.370426 8450 tablet_service.cc:1514] Processing DeleteTablet for tablet 435e97b527ae477b8dbe318ff1dbd135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:17 UTC) from {username='slave'} at 127.0.0.1:39222
I20260925 09:20:17.370628 8189 tablet_service.cc:1514] Processing DeleteTablet for tablet 435e97b527ae477b8dbe318ff1dbd135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:17 UTC) from {username='slave'} at 127.0.0.1:60420
I20260925 09:20:17.374274 8320 tablet_service.cc:1514] Processing DeleteTablet for tablet 435e97b527ae477b8dbe318ff1dbd135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:17 UTC) from {username='slave'} at 127.0.0.1:59664
I20260925 09:20:17.378439 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8127
I20260925 09:20:17.379853 8559 tablet_replica.cc:330] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: stopping tablet replica
I20260925 09:20:17.380319 8559 raft_consensus.cc:2229] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:17.380872 8559 raft_consensus.cc:2258] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:17.383984 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.379383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.3.82.1:46063)
0925 09:20:17.379588 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:17.379594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:17.381571 (+ 1977us) server_negotiation.cc:407] Connection header received
0925 09:20:17.381940 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:17.381948 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:17.382039 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:17.382179 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:17.383194 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:17.383200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:17.383205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:17.383223 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:17.383258 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:17.383273 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:17.383383 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:17.383611 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 09:20:17.383677 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:20:17.385089 8249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:17.390388 8557 tablet_replica.cc:330] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: stopping tablet replica
I20260925 09:20:17.390903 8557 raft_consensus.cc:2229] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:17.391449 8557 raft_consensus.cc:2258] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:17.393781 8557 ts_tablet_manager.cc:1905] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:17.397231 8559 ts_tablet_manager.cc:1905] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:17.402160 8557 ts_tablet_manager.cc:1918] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:17.402354 8557 log.cc:1196] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/435e97b527ae477b8dbe318ff1dbd135
I20260925 09:20:17.402866 8557 ts_tablet_manager.cc:1939] T 435e97b527ae477b8dbe318ff1dbd135 P 90f1571ada7b443491c960ffc65f6954: Deleting consensus metadata
I20260925 09:20:17.404902 8054 catalog_manager.cc:4841] TS 90f1571ada7b443491c960ffc65f6954 (127.3.82.3:42545): tablet 435e97b527ae477b8dbe318ff1dbd135 (table test-table [id=c5f00fb51511440d9f64c53070e50250]) successfully deleted
I20260925 09:20:17.407405 8559 ts_tablet_manager.cc:1918] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:17.407608 8559 log.cc:1196] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/435e97b527ae477b8dbe318ff1dbd135
I20260925 09:20:17.408201 8559 ts_tablet_manager.cc:1939] T 435e97b527ae477b8dbe318ff1dbd135 P 4fb3f29e4816415abcba39cd21e42e76: Deleting consensus metadata
I20260925 09:20:17.410591 8052 catalog_manager.cc:4841] TS 4fb3f29e4816415abcba39cd21e42e76 (127.3.82.2:46521): tablet 435e97b527ae477b8dbe318ff1dbd135 (table test-table [id=c5f00fb51511440d9f64c53070e50250]) successfully deleted
I20260925 09:20:17.499874 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8127
I20260925 09:20:17.499980 8558 tablet_replica.cc:330] T 435e97b527ae477b8dbe318ff1dbd135 P 39824e91c45444df899361792be7411b: stopping tablet replica
W20260925 09:20:17.514111 8052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.82.1:46063 (error 108)
I20260925 09:20:17.514868 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8258
I20260925 09:20:17.516440 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.515538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60444)
0925 09:20:17.516170 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.516284 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":417,"threads_started":1}
W20260925 09:20:17.516752 8052 catalog_manager.cc:4584] TS 39824e91c45444df899361792be7411b (127.3.82.1:46063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 435e97b527ae477b8dbe318ff1dbd135: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
I20260925 09:20:17.518126 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.515719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.3.82.2:46521)
0925 09:20:17.515949 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:17.515956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:17.515978 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:20:17.516133 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:17.516139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:17.516279 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:17.516400 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:17.517458 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:17.517464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:17.517469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:17.517490 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:17.517530 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:17.517542 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:17.517682 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:17.517763 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 09:20:17.517828 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:20:17.518932 8380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:17.570946 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.570321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60452)
0925 09:20:17.570591 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.570755 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260925 09:20:17.572257 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.571815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60458)
0925 09:20:17.571998 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.572087 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 09:20:17.611944 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8258
I20260925 09:20:17.629626 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8388
I20260925 09:20:17.632604 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.632085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60468)
0925 09:20:17.632377 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.632456 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 09:20:17.632979 8531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.630526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39238 (local address 127.3.82.3:42545)
0925 09:20:17.630724 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:17.630730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:17.630835 (+ 105us) server_negotiation.cc:407] Connection header received
0925 09:20:17.631084 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:17.631095 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:17.631190 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:17.631303 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:17.632248 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:17.632259 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:17.632262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:17.632278 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:17.632310 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:17.632322 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:17.632535 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:17.632615 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 09:20:17.632679 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:20:17.633736 8510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:17.633810 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.633318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60472)
0925 09:20:17.633515 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.633623 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 09:20:17.714942 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.714248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60488)
0925 09:20:17.714668 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.714773 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260925 09:20:17.716100 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:17.715619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46063 (local address 127.0.0.1:60496)
0925 09:20:17.715828 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:17.715927 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:46063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260925 09:20:17.724961 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8388
I20260925 09:20:17.741722 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8037
I20260925 09:20:17.743211 8098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:17.855123 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8037
2026-09-25T09:20:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2994 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 09:20:17.897209 3400 mini_oidc.cc:152] Starting JWKS server
I20260925 09:20:17.899915 3400 webserver.cc:460] Webserver started at https://127.0.0.1:38643/ using document root <none> and password file <none>
I20260925 09:20:17.900270 3400 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 09:20:17.901000 3400 webserver.cc:460] Webserver started at http://127.0.0.1:44625/ using document root <none> and password file <none>
2026-09-25T09:20:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:17Z Disabled control of system clock
I20260925 09:20:17.930267 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:34329
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:34329
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38643/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/testchainca.cert with env {}
W20260925 09:20:18.269960 8572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:18.270341 8572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:18.270520 8572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:18.278028 8572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:18.278214 8572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:18.278311 8572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:18.278412 8572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:18.278497 8572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:20:18.278587 8572 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:20:18.278648 8572 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38643/jwks.json
I20260925 09:20:18.289067 8572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:34329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38643/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:18.290546 8572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:18.291939 8572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:18.304520 8580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:18.304591 8577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:18.304548 8578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:18.310708 8572 server_base.cc:1034] running on GCE node
I20260925 09:20:18.311703 8572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:18.313371 8572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:18.314618 8572 hybrid_clock.cc:648] HybridClock initialized: now 1790328018314556 us; error 64 us; skew 500 ppm
I20260925 09:20:18.315004 8572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:18.323348 8572 webserver.cc:460] Webserver started at http://127.3.82.62:43145/ using document root <none> and password file <none>
I20260925 09:20:18.324174 8572 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:18.324314 8572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:18.324663 8572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:18.327904 8572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "f207c5fec9be4bdba1b5abf6d14d89d0"
format_stamp: "Formatted at 2026-09-25 09:20:18 on dist-test-slave-4wm5"
server_key: "1f6880c0e111f8f548df6846c0c483da"
server_key_iv: "8e8e4f5d6582281db1622d56ec23ebbd"
server_key_version: "encryptionkey@0"
I20260925 09:20:18.328761 8572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "f207c5fec9be4bdba1b5abf6d14d89d0"
format_stamp: "Formatted at 2026-09-25 09:20:18 on dist-test-slave-4wm5"
server_key: "1f6880c0e111f8f548df6846c0c483da"
server_key_iv: "8e8e4f5d6582281db1622d56ec23ebbd"
server_key_version: "encryptionkey@0"
I20260925 09:20:18.334852 8572 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260925 09:20:18.338835 8586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:18.340556 8572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:20:18.340811 8572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "f207c5fec9be4bdba1b5abf6d14d89d0"
format_stamp: "Formatted at 2026-09-25 09:20:18 on dist-test-slave-4wm5"
server_key: "1f6880c0e111f8f548df6846c0c483da"
server_key_iv: "8e8e4f5d6582281db1622d56ec23ebbd"
server_key_version: "encryptionkey@0"
I20260925 09:20:18.341092 8572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:18.359982 8572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:18.361048 8572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:18.361375 8572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:18.385426 8572 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:34329
I20260925 09:20:18.385444 8637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:34329 every 8 connection(s)
I20260925 09:20:18.387157 8572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:18.391826 8638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:18.392004 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8572
I20260925 09:20:18.392221 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:18.393028 3400 external_mini_cluster.cc:1428] Setting key 3542aaeacb3bd2df62f5426ceaeea9f0
I20260925 09:20:18.407142 8638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0: Bootstrap starting.
I20260925 09:20:18.409263 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:18.395247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.3.82.62:34329)
0925 09:20:18.396536 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:18.396548 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:18.396596 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:20:18.398768 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:18.398796 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:18.399422 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:18.399808 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:18.400540 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:18.402101 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:18.404163 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:18.405008 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:18.405153 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:18.406701 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:18.406708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:18.406725 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:18.407381 (+ 656us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:18.407468 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:18.407491 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:18.407601 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:18.407842 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 09:20:18.408211 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":558,"threads_started":1}
I20260925 09:20:18.410849 8638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:18.412127 8638 log.cc:824] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:18.413595 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:20:18.415189 8638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0: No bootstrap required, opened a new log
I20260925 09:20:18.422724 8638 raft_consensus.cc:348] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34329 } }
I20260925 09:20:18.423208 8638 raft_consensus.cc:374] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:18.423352 8638 raft_consensus.cc:729] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f207c5fec9be4bdba1b5abf6d14d89d0, State: Initialized, Role: FOLLOWER
I20260925 09:20:18.424012 8638 consensus_queue.cc:260] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34329 } }
I20260925 09:20:18.424242 8638 raft_consensus.cc:388] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:18.424345 8638 raft_consensus.cc:482] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:18.424479 8638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:18.426752 8638 raft_consensus.cc:504] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34329 } }
I20260925 09:20:18.427350 8638 leader_election.cc:302] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f207c5fec9be4bdba1b5abf6d14d89d0; no voters:
I20260925 09:20:18.428004 8638 leader_election.cc:288] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:18.428217 8643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:18.429636 8643 raft_consensus.cc:686] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [term 1 LEADER]: Becoming Leader. State: Replica: f207c5fec9be4bdba1b5abf6d14d89d0, State: Running, Role: LEADER
I20260925 09:20:18.430367 8643 consensus_queue.cc:237] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34329 } }
I20260925 09:20:18.430737 8638 sys_catalog.cc:564] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:18.433868 8645 sys_catalog.cc:455] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f207c5fec9be4bdba1b5abf6d14d89d0. Latest consensus state: current_term: 1 leader_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34329 } } }
I20260925 09:20:18.434377 8645 sys_catalog.cc:458] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:18.434456 8644 sys_catalog.cc:455] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f207c5fec9be4bdba1b5abf6d14d89d0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34329 } } }
I20260925 09:20:18.434839 8644 sys_catalog.cc:458] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:18.436856 8650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:18.442736 8650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:18.453470 8650 catalog_manager.cc:1347] Generated new cluster ID: 0478a9525483488c8e122f123b0c010c
I20260925 09:20:18.453611 8650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:18.475672 8650 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:18.477085 8650 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:18.493039 8650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f207c5fec9be4bdba1b5abf6d14d89d0: Generated new TSK 0
I20260925 09:20:18.494216 8650 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 09:20:18.532125 8668 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 09:20:18.532348 8668 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 09:20:18.533499 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:18.511005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.3.82.62:34329)
0925 09:20:18.511178 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:18.511183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:18.511675 (+ 492us) server_negotiation.cc:407] Connection header received
0925 09:20:18.511993 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:18.511999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:18.512102 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:18.512277 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:20:18.512867 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:18.514051 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:18.515578 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:18.516308 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:18.516442 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:18.517073 (+ 631us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:20:18.532761 (+ 15688us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 09:20:18.533189 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 09:20:18.533640 8641 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 09:20:18.534974 8667 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:18.510845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34329 (local address 127.0.0.1:44062)
0925 09:20:18.511553 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:18.511570 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:18.511732 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:18.512320 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:18.512324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:18.512334 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:20:18.512683 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:18.512692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:18.514195 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:18.514199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:18.515373 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:18.515385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:18.516477 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:18.516537 (+ 60us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:18.516670 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:18.516739 (+ 69us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:20:18.533076 (+ 16337us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 09:20:18.533366 (+ 290us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.82.62:34329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":354,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.82.62:34329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 09:20:18.537129 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8572
I20260925 09:20:18.538568 8633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:18.638928 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8572
2026-09-25T09:20:18Z chronyd exiting
I20260925 09:20:18.690028 3400 test_util.cc:175] -----------------------------------------------
I20260925 09:20:18.690213 3400 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790327973894689-3400-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (797 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 09:20:18.694217 3400 mini_oidc.cc:152] Starting JWKS server
I20260925 09:20:18.696099 3400 webserver.cc:460] Webserver started at https://127.0.0.1:36025/ using document root <none> and password file <none>
I20260925 09:20:18.696363 3400 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 09:20:18.697072 3400 webserver.cc:460] Webserver started at http://127.0.0.1:40697/ using document root <none> and password file <none>
2026-09-25T09:20:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:18Z Disabled control of system clock
I20260925 09:20:18.726675 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45061
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45061
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36025/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/testchainca.cert with env {}
W20260925 09:20:19.073457 8675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:19.073774 8675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:19.073973 8675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:19.081580 8675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:19.081712 8675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:19.081750 8675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:19.081802 8675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:19.081913 8675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:20:19.082003 8675 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:20:19.082039 8675 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36025/jwks.json
I20260925 09:20:19.092530 8675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45061
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36025/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:19.094022 8675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:19.095376 8675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:19.108017 8680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:19.107954 8681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:19.108184 8683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:19.108809 8675 server_base.cc:1034] running on GCE node
I20260925 09:20:19.109892 8675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:19.111485 8675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:19.112701 8675 hybrid_clock.cc:648] HybridClock initialized: now 1790328019112636 us; error 69 us; skew 500 ppm
I20260925 09:20:19.113137 8675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:19.115644 8675 webserver.cc:460] Webserver started at http://127.3.82.62:43739/ using document root <none> and password file <none>
I20260925 09:20:19.116393 8675 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:19.116511 8675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:19.116870 8675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:19.119972 8675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "23cf121d881b41a794f94adbe274f99c"
format_stamp: "Formatted at 2026-09-25 09:20:19 on dist-test-slave-4wm5"
server_key: "329b90410e2f2f13ed70f66ff9a6e877"
server_key_iv: "4cf7d84b8198157dca7310d9f1153a8b"
server_key_version: "encryptionkey@0"
I20260925 09:20:19.120846 8675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "23cf121d881b41a794f94adbe274f99c"
format_stamp: "Formatted at 2026-09-25 09:20:19 on dist-test-slave-4wm5"
server_key: "329b90410e2f2f13ed70f66ff9a6e877"
server_key_iv: "4cf7d84b8198157dca7310d9f1153a8b"
server_key_version: "encryptionkey@0"
I20260925 09:20:19.126394 8675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:20:19.130215 8689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:19.131743 8675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:20:19.131986 8675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "23cf121d881b41a794f94adbe274f99c"
format_stamp: "Formatted at 2026-09-25 09:20:19 on dist-test-slave-4wm5"
server_key: "329b90410e2f2f13ed70f66ff9a6e877"
server_key_iv: "4cf7d84b8198157dca7310d9f1153a8b"
server_key_version: "encryptionkey@0"
I20260925 09:20:19.132222 8675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:19.153990 8675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:19.154959 8675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:19.155267 8675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:19.173977 8675 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45061
I20260925 09:20:19.174000 8740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45061 every 8 connection(s)
I20260925 09:20:19.175490 8675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:19.178830 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8675
I20260925 09:20:19.179056 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:19.179680 8741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:19.179807 3400 external_mini_cluster.cc:1428] Setting key 18b1ba6b24050539c75adc45d38cc25d
I20260925 09:20:19.195809 8741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c: Bootstrap starting.
I20260925 09:20:19.196095 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:19.181597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.3.82.62:45061)
0925 09:20:19.183073 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:19.183088 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:19.183142 (+ 54us) server_negotiation.cc:407] Connection header received
0925 09:20:19.185133 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:19.185164 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:19.185863 (+ 699us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:19.186390 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:19.187069 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.189098 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.190601 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.191679 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.191890 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:19.193059 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:19.193071 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:19.193098 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:19.193898 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:19.194008 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:19.194034 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:19.194154 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:19.194415 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 09:20:19.194759 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":720,"threads_started":1}
I20260925 09:20:19.199491 8741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:19.200359 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:20:19.200697 8741 log.cc:824] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:19.203945 8741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c: No bootstrap required, opened a new log
I20260925 09:20:19.211843 8741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cf121d881b41a794f94adbe274f99c" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45061 } }
I20260925 09:20:19.212235 8741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:19.212317 8741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cf121d881b41a794f94adbe274f99c, State: Initialized, Role: FOLLOWER
I20260925 09:20:19.212847 8741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cf121d881b41a794f94adbe274f99c" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45061 } }
I20260925 09:20:19.213052 8741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:19.213150 8741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:19.213272 8741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:19.215503 8741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cf121d881b41a794f94adbe274f99c" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45061 } }
I20260925 09:20:19.216032 8741 leader_election.cc:302] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23cf121d881b41a794f94adbe274f99c; no voters:
I20260925 09:20:19.216585 8741 leader_election.cc:288] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:19.216822 8746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:19.218209 8746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [term 1 LEADER]: Becoming Leader. State: Replica: 23cf121d881b41a794f94adbe274f99c, State: Running, Role: LEADER
I20260925 09:20:19.218856 8746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cf121d881b41a794f94adbe274f99c" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45061 } }
I20260925 09:20:19.219043 8741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:19.222198 8748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23cf121d881b41a794f94adbe274f99c. Latest consensus state: current_term: 1 leader_uuid: "23cf121d881b41a794f94adbe274f99c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cf121d881b41a794f94adbe274f99c" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45061 } } }
I20260925 09:20:19.223383 8748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:19.222400 8747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23cf121d881b41a794f94adbe274f99c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cf121d881b41a794f94adbe274f99c" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45061 } } }
I20260925 09:20:19.224113 8747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:19.225806 8755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:19.231750 8755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:19.242656 8755 catalog_manager.cc:1347] Generated new cluster ID: 821da04f6264470bad53c891845dd3c0
I20260925 09:20:19.242786 8755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:19.274284 8755 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:19.275792 8755 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:19.290473 8755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23cf121d881b41a794f94adbe274f99c: Generated new TSK 0
I20260925 09:20:19.291431 8755 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 09:20:19.332746 8771 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 09:20:19.332902 8771 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 09:20:19.334568 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:19.311130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.3.82.62:45061)
0925 09:20:19.311412 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:19.311418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:19.311803 (+ 385us) server_negotiation.cc:407] Connection header received
0925 09:20:19.312098 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:19.312103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:19.312200 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:19.312447 (+ 247us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:20:19.313013 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.314031 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.316423 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.316938 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.317068 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:19.317795 (+ 727us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:20:19.333814 (+ 16019us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 09:20:19.334291 (+ 477us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41610: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 09:20:19.334749 8744 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41610: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 09:20:19.334966 8770 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:19.310994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45061 (local address 127.0.0.1:41610)
0925 09:20:19.311660 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:19.311681 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:19.311865 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:19.312493 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:19.312498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:19.312512 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:20:19.312844 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:19.312857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.314199 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.314205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:19.316096 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:19.316121 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.317105 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.317166 (+ 61us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:19.317372 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:19.317427 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:20:19.334175 (+ 16748us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 09:20:19.334483 (+ 308us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.82.62:45061: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36025/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":357,"threads_started":1}
I20260925 09:20:19.336179 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8675
I20260925 09:20:19.337658 8736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:19.435272 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8675
2026-09-25T09:20:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (797 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 09:20:19.491603 3400 mini_oidc.cc:152] Starting JWKS server
I20260925 09:20:19.493603 3400 webserver.cc:460] Webserver started at https://127.0.0.1:38767/ using document root <none> and password file <none>
I20260925 09:20:19.493937 3400 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 09:20:19.494630 3400 webserver.cc:460] Webserver started at http://127.0.0.1:46867/ using document root <none> and password file <none>
2026-09-25T09:20:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:19Z Disabled control of system clock
I20260925 09:20:19.525153 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:41867
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:41867
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38767/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 09:20:19.866698 8778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:19.867038 8778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:19.867127 8778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:19.874758 8778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:19.874886 8778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:19.874924 8778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:19.874974 8778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:19.875077 8778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:20:19.875180 8778 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:20:19.875227 8778 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38767/jwks.json
I20260925 09:20:19.884923 8778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:41867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:41867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38767/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:19.886410 8778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:19.888355 8778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:19.901527 8786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:19.901628 8783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:19.901635 8784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:19.902832 8778 server_base.cc:1034] running on GCE node
I20260925 09:20:19.903673 8778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:19.905205 8778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:19.908574 8778 hybrid_clock.cc:648] HybridClock initialized: now 1790328019908485 us; error 71 us; skew 500 ppm
I20260925 09:20:19.909039 8778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:19.911828 8778 webserver.cc:460] Webserver started at http://127.3.82.62:33621/ using document root <none> and password file <none>
I20260925 09:20:19.912637 8778 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:19.912734 8778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:19.913121 8778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:19.916330 8778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e"
format_stamp: "Formatted at 2026-09-25 09:20:19 on dist-test-slave-4wm5"
server_key: "27e33998130fa5ab52419adf8c7d1323"
server_key_iv: "043da7d7283909e88eae62efc7a413ef"
server_key_version: "encryptionkey@0"
I20260925 09:20:19.917270 8778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e"
format_stamp: "Formatted at 2026-09-25 09:20:19 on dist-test-slave-4wm5"
server_key: "27e33998130fa5ab52419adf8c7d1323"
server_key_iv: "043da7d7283909e88eae62efc7a413ef"
server_key_version: "encryptionkey@0"
I20260925 09:20:19.923599 8778 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260925 09:20:19.928252 8792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:19.930078 8778 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:19.930310 8778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e"
format_stamp: "Formatted at 2026-09-25 09:20:19 on dist-test-slave-4wm5"
server_key: "27e33998130fa5ab52419adf8c7d1323"
server_key_iv: "043da7d7283909e88eae62efc7a413ef"
server_key_version: "encryptionkey@0"
I20260925 09:20:19.930572 8778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:19.948729 8778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:19.949705 8778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:19.950043 8778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:19.969663 8778 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:41867
I20260925 09:20:19.969684 8843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:41867 every 8 connection(s)
I20260925 09:20:19.971201 8778 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:19.975765 8844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:19.977465 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8778
I20260925 09:20:19.977710 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:19.978387 3400 external_mini_cluster.cc:1428] Setting key 0dc913b239258f81786bb0f5a6573909
I20260925 09:20:19.990016 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e: Bootstrap starting.
I20260925 09:20:19.994397 8844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:19.995548 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:19.980532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.3.82.62:41867)
0925 09:20:19.981766 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:19.981787 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:19.981863 (+ 76us) server_negotiation.cc:407] Connection header received
0925 09:20:19.984287 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:19.984317 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:19.984932 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:19.985400 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:19.986052 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.988185 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.989660 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:19.990760 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:19.990947 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:19.992309 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:19.992321 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:19.992343 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:19.993131 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:19.993277 (+ 146us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:19.993308 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:19.993427 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:19.993702 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 09:20:19.994102 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":390,"threads_started":1}
I20260925 09:20:19.995857 8844 log.cc:824] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:19.999719 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e: No bootstrap required, opened a new log
I20260925 09:20:20.000968 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:20:20.008994 8844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 41867 } }
I20260925 09:20:20.009555 8844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:20.009709 8844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01191d914ecf4f9bbe4e8a4cfa92cb9e, State: Initialized, Role: FOLLOWER
I20260925 09:20:20.010383 8844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 41867 } }
I20260925 09:20:20.010608 8844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:20.010726 8844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:20.010872 8844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:20.013118 8844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 41867 } }
I20260925 09:20:20.013656 8844 leader_election.cc:302] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01191d914ecf4f9bbe4e8a4cfa92cb9e; no voters:
I20260925 09:20:20.014377 8844 leader_election.cc:288] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:20.014555 8849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:20.016103 8849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [term 1 LEADER]: Becoming Leader. State: Replica: 01191d914ecf4f9bbe4e8a4cfa92cb9e, State: Running, Role: LEADER
I20260925 09:20:20.016687 8849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 41867 } }
I20260925 09:20:20.017218 8844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:20.019646 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01191d914ecf4f9bbe4e8a4cfa92cb9e. Latest consensus state: current_term: 1 leader_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 41867 } } }
I20260925 09:20:20.020067 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:20.020689 8850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01191d914ecf4f9bbe4e8a4cfa92cb9e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 41867 } } }
I20260925 09:20:20.021101 8850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:20.021116 8858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:20.030990 8858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:20.041673 8858 catalog_manager.cc:1347] Generated new cluster ID: e322be00a880422da2b1acc8b155276d
I20260925 09:20:20.041815 8858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:20.072238 8858 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:20.073686 8858 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:20.084858 8858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01191d914ecf4f9bbe4e8a4cfa92cb9e: Generated new TSK 0
I20260925 09:20:20.085809 8858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:20:20.120988 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:20.104938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.3.82.62:41867)
0925 09:20:20.105192 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:20.105199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:20.105598 (+ 399us) server_negotiation.cc:407] Connection header received
0925 09:20:20.105999 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:20.106004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:20.106092 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:20.106231 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:20:20.106795 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.107562 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.109754 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.110328 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.110440 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:20.111086 (+ 646us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:20:20.120055 (+ 8969us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38767/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 09:20:20.120564 (+ 509us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42470: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38767/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":88}
W20260925 09:20:20.121150 8847 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42470: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38767/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 09:20:20.122396 8874 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:20.104744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:41867 (local address 127.0.0.1:42470)
0925 09:20:20.105475 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:20.105495 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:20.105659 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:20.106281 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:20.106287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:20.106301 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:20:20.106617 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:20.106631 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.107724 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.107730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:20.109580 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:20.109597 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.110458 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.110555 (+ 97us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:20.110748 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:20.110800 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:20:20.121486 (+ 10686us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38767/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 09:20:20.121849 (+ 363us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.82.62:41867: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38767/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":383,"threads_started":1}
I20260925 09:20:20.123735 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8778
I20260925 09:20:20.125116 8839 generic_service.cc:196] Checking for leaks (request via RPC)
E20260925 09:20:20.135975 8875 webserver.cc:533] Webserver: sslize failed:
E20260925 09:20:20.136080 8875 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 09:20:20.225162 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8778
2026-09-25T09:20:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (789 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T09:20:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:20Z Disabled control of system clock
I20260925 09:20:20.309460 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42545
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42545
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/testchainca.cert with env {}
W20260925 09:20:20.652541 8880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:20.652868 8880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:20.653000 8880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:20.660624 8880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:20.660745 8880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:20.660815 8880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:20.660899 8880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:20.660959 8880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 09:20:20.661091 8880 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 09:20:20.661149 8880 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 09:20:20.671780 8880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:20.673215 8880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:20.674616 8880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:20.687095 8886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:20.687063 8885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:20.687172 8888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:20.688153 8880 server_base.cc:1034] running on GCE node
I20260925 09:20:20.689064 8880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:20.690791 8880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:20.692026 8880 hybrid_clock.cc:648] HybridClock initialized: now 1790328020691968 us; error 49 us; skew 500 ppm
I20260925 09:20:20.692436 8880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:20.694986 8880 webserver.cc:460] Webserver started at http://127.3.82.62:44079/ using document root <none> and password file <none>
I20260925 09:20:20.695758 8880 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:20.695863 8880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:20.696187 8880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:20.699482 8880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "1a64075fd09d4679ac64d7d70730a338"
format_stamp: "Formatted at 2026-09-25 09:20:20 on dist-test-slave-4wm5"
server_key: "3c04bf2d4998cfd5987023fccf0d668a"
server_key_iv: "0afc404d5b9a636300a2911e09753adc"
server_key_version: "encryptionkey@0"
I20260925 09:20:20.700410 8880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "1a64075fd09d4679ac64d7d70730a338"
format_stamp: "Formatted at 2026-09-25 09:20:20 on dist-test-slave-4wm5"
server_key: "3c04bf2d4998cfd5987023fccf0d668a"
server_key_iv: "0afc404d5b9a636300a2911e09753adc"
server_key_version: "encryptionkey@0"
I20260925 09:20:20.705811 8880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:20:20.710047 8894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:20.711957 8880 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:20:20.712221 8880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "1a64075fd09d4679ac64d7d70730a338"
format_stamp: "Formatted at 2026-09-25 09:20:20 on dist-test-slave-4wm5"
server_key: "3c04bf2d4998cfd5987023fccf0d668a"
server_key_iv: "0afc404d5b9a636300a2911e09753adc"
server_key_version: "encryptionkey@0"
I20260925 09:20:20.712481 8880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:20.735496 8880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:20.736442 8880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:20.736752 8880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:20.757179 8880 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:42545
I20260925 09:20:20.757232 8945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:42545 every 8 connection(s)
I20260925 09:20:20.758744 8880 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:20.761718 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 8880
I20260925 09:20:20.761991 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:20.762786 3400 external_mini_cluster.cc:1428] Setting key 162e950763b2e5ffb25a09d6e5274ca0
I20260925 09:20:20.763643 8946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:20.781327 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:20.765014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.3.82.62:42545)
0925 09:20:20.766260 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:20.766276 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:20.766336 (+ 60us) server_negotiation.cc:407] Connection header received
0925 09:20:20.769098 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:20.769135 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:20.769931 (+ 796us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:20.770512 (+ 581us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:20.771181 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.773504 (+ 2323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.775137 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.776325 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.776548 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:20.777913 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:20.777925 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:20.777949 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:20:20.778825 (+ 876us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:20.778948 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:20:20.778983 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:20.779115 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:20.779408 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 09:20:20.779801 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":379,"threads_started":1}
I20260925 09:20:20.781327 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338: Bootstrap starting.
I20260925 09:20:20.785425 8946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:20.785876 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:20:20.786814 8946 log.cc:824] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:20.790727 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338: No bootstrap required, opened a new log
I20260925 09:20:20.799566 8946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a64075fd09d4679ac64d7d70730a338" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42545 } }
I20260925 09:20:20.799989 8946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:20.800079 8946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a64075fd09d4679ac64d7d70730a338, State: Initialized, Role: FOLLOWER
I20260925 09:20:20.800648 8946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a64075fd09d4679ac64d7d70730a338" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42545 } }
I20260925 09:20:20.800879 8946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:20.800995 8946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:20.801133 8946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:20.803920 8946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a64075fd09d4679ac64d7d70730a338" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42545 } }
I20260925 09:20:20.804570 8946 leader_election.cc:302] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a64075fd09d4679ac64d7d70730a338; no voters:
I20260925 09:20:20.805256 8946 leader_election.cc:288] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:20.805531 8951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:20.806357 8951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [term 1 LEADER]: Becoming Leader. State: Replica: 1a64075fd09d4679ac64d7d70730a338, State: Running, Role: LEADER
I20260925 09:20:20.806984 8951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a64075fd09d4679ac64d7d70730a338" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42545 } }
I20260925 09:20:20.807698 8946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:20.810089 8953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a64075fd09d4679ac64d7d70730a338. Latest consensus state: current_term: 1 leader_uuid: "1a64075fd09d4679ac64d7d70730a338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a64075fd09d4679ac64d7d70730a338" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42545 } } }
I20260925 09:20:20.810130 8952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a64075fd09d4679ac64d7d70730a338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a64075fd09d4679ac64d7d70730a338" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42545 } } }
I20260925 09:20:20.810814 8952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:20.812024 8953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:20.815198 8959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:20.821151 8959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:20.831311 8959 catalog_manager.cc:1347] Generated new cluster ID: 904d0b8854504fd7af0dac2e4eef0fd2
I20260925 09:20:20.831465 8959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:20.861157 8959 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:20.863147 8959 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:20.880898 8959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a64075fd09d4679ac64d7d70730a338: Generated new TSK 0
I20260925 09:20:20.882227 8959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:20:21.395498 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:20.910936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37382 (local address 127.3.82.62:42545)
0925 09:20:20.911174 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:20.911181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:20.911745 (+ 564us) server_negotiation.cc:407] Connection header received
0925 09:20:20.912013 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:20.912019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:20.912128 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:20.912307 (+ 179us) server_negotiation.cc:226] Negotiated authn=JWT
0925 09:20:20.912800 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.914086 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.915592 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.916379 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.916525 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:20.917503 (+ 978us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 09:20:21.394863 (+477360us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 09:20:21.395238 (+ 375us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37382: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":91}
W20260925 09:20:21.395612 8949 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37382: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 09:20:21.396042 8975 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:20.910839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42545 (local address 127.0.0.1:37382)
0925 09:20:20.911630 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:20.911647 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:20.911794 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:20.912322 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:20.912326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:20.912335 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0925 09:20:20.912595 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:20.912604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.914255 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.914260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:20.915432 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:20.915446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:20.916577 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:20.916685 (+ 108us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:20.916965 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:20.917036 (+ 71us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 09:20:21.395198 (+478162us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 09:20:21.395549 (+ 351us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.82.62:42545: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":436,"threads_started":1}
I20260925 09:20:21.397176 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 8880
I20260925 09:20:21.398423 8941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:21.506680 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 8880
2026-09-25T09:20:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1263 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (447 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1567 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:23 dist-test-slave-4wm5 krb5kdc[8991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:23 dist-test-slave-4wm5 krb5kdc[8991](info): set up 2 sockets
Sep 25 09:20:23 dist-test-slave-4wm5 krb5kdc[8991](info): commencing operation
krb5kdc: starting...
W20260925 09:20:24.634886 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:24 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:24.869941 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:34015
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:34015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:25.224409 9004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:25.224805 9004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:25.224984 9004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:25.232518 9004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:25.232718 9004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:25.232829 9004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:25.232932 9004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:25.233003 9004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:25.243417 9004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:34015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:34015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:25.245224 9004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:25.246754 9004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:25.260177 9009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:25.260170 9012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:25.260179 9010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:25.261190 9004 server_base.cc:1034] running on GCE node
I20260925 09:20:25.262212 9004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:25.263935 9004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:25.265178 9004 hybrid_clock.cc:648] HybridClock initialized: now 1790328025265126 us; error 40 us; skew 500 ppm
I20260925 09:20:25.265657 9004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:25 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328025, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:25.269243 9004 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:25.271799 9004 webserver.cc:460] Webserver started at http://127.3.82.62:43771/ using document root <none> and password file <none>
I20260925 09:20:25.272737 9004 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:25.272886 9004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:25.273298 9004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:25.276610 9004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "5c351a4f761c42f4bd7f7814f2ad6e24"
format_stamp: "Formatted at 2026-09-25 09:20:25 on dist-test-slave-4wm5"
server_key: "df781d623e536e71cf80c811db3d6b74"
server_key_iv: "74866a81849d1dd57ec7b62a0aab18de"
server_key_version: "encryptionkey@0"
I20260925 09:20:25.277542 9004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "5c351a4f761c42f4bd7f7814f2ad6e24"
format_stamp: "Formatted at 2026-09-25 09:20:25 on dist-test-slave-4wm5"
server_key: "df781d623e536e71cf80c811db3d6b74"
server_key_iv: "74866a81849d1dd57ec7b62a0aab18de"
server_key_version: "encryptionkey@0"
I20260925 09:20:25.283381 9004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:20:25.287705 9019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:25.289717 9004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260925 09:20:25.289965 9004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "5c351a4f761c42f4bd7f7814f2ad6e24"
format_stamp: "Formatted at 2026-09-25 09:20:25 on dist-test-slave-4wm5"
server_key: "df781d623e536e71cf80c811db3d6b74"
server_key_iv: "74866a81849d1dd57ec7b62a0aab18de"
server_key_version: "encryptionkey@0"
I20260925 09:20:25.290194 9004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:25.307722 9004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:25.312386 9004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:25.312704 9004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:25.332859 9004 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:34015
I20260925 09:20:25.332955 9070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:34015 every 8 connection(s)
I20260925 09:20:25.334355 9004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:25.335932 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9004
I20260925 09:20:25.336140 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:25.336759 3400 external_mini_cluster.cc:1428] Setting key f55237481479445be5aae23bf117415e
I20260925 09:20:25.338812 9071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:20:25 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:25.351130 9071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24: Bootstrap starting.
I20260925 09:20:25.355116 9071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:25.356261 9071 log.cc:824] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:25.359457 9071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24: No bootstrap required, opened a new log
I20260925 09:20:25.360069 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:25.338830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.3.82.62:34015)
0925 09:20:25.339585 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:25.339598 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:25.339640 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:20:25.341273 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:25.341295 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:25.341721 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:25.342264 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:25.343786 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:25.345166 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:25.346522 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:25.347267 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:25.347483 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:25.352190 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:25.352199 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:25.352217 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:25.352259 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:25.355679 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:25.356462 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:25.356469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:25.356476 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:25.356650 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:25.357086 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:25.357090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:25.357092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:25.357672 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:25.358026 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:25.358545 (+ 519us) server_negotiation.cc:299] Negotiation successful
0925 09:20:25.358990 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":331,"threads_started":1}
I20260925 09:20:25.368288 9071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34015 } }
I20260925 09:20:25.368755 9071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:25.368883 9071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c351a4f761c42f4bd7f7814f2ad6e24, State: Initialized, Role: FOLLOWER
I20260925 09:20:25.369581 9071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34015 } }
I20260925 09:20:25.369913 9071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:25.370078 9071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:25.370282 9071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:25.372996 9071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34015 } }
I20260925 09:20:25.373708 9071 leader_election.cc:302] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c351a4f761c42f4bd7f7814f2ad6e24; no voters:
I20260925 09:20:25.374425 9071 leader_election.cc:288] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:25.374599 9076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:25.375331 9076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [term 1 LEADER]: Becoming Leader. State: Replica: 5c351a4f761c42f4bd7f7814f2ad6e24, State: Running, Role: LEADER
I20260925 09:20:25.375926 9076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34015 } }
I20260925 09:20:25.376639 9071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:25.379281 9077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34015 } } }
I20260925 09:20:25.379191 9078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c351a4f761c42f4bd7f7814f2ad6e24. Latest consensus state: current_term: 1 leader_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c351a4f761c42f4bd7f7814f2ad6e24" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 34015 } } }
I20260925 09:20:25.379727 9077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:25.379746 9078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:25.380887 9082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:25.387698 9082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:25.398435 9082 catalog_manager.cc:1347] Generated new cluster ID: 749d5525982e426b8acc3e8ed3c72e61
I20260925 09:20:25.398551 9082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:25.421020 9082 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:25.422530 9082 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:25.431923 9082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c351a4f761c42f4bd7f7814f2ad6e24: Generated new TSK 0
I20260925 09:20:25.432981 9082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:20:25.494771 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34015
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:25.843712 9097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:25.844023 9097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:25.844202 9097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:25.851701 9097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:25.851830 9097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:25.852033 9097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:25.862434 9097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:25.863823 9097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:25.865180 9097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:25.879534 9102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:25.879752 9103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:25.880285 9105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:25.881778 9097 server_base.cc:1034] running on GCE node
I20260925 09:20:25.882586 9097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:25.883731 9097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:25.885030 9097 hybrid_clock.cc:648] HybridClock initialized: now 1790328025884969 us; error 75 us; skew 500 ppm
I20260925 09:20:25.885480 9097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:25 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328025, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:25.889196 9097 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:20:25.891064 9097 webserver.cc:460] Webserver started at http://127.3.82.1:38637/ using document root <none> and password file <none>
I20260925 09:20:25.891820 9097 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:25.891973 9097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:25.892305 9097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:25.895712 9097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "604d6f6b0eb14b33b2b95f1149dc1c4a"
format_stamp: "Formatted at 2026-09-25 09:20:25 on dist-test-slave-4wm5"
server_key: "ede2717a5b4b22663545ae80b34c4a5b"
server_key_iv: "7ff47261cdf7d825017eaac1a37f6095"
server_key_version: "encryptionkey@0"
I20260925 09:20:25.896569 9097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "604d6f6b0eb14b33b2b95f1149dc1c4a"
format_stamp: "Formatted at 2026-09-25 09:20:25 on dist-test-slave-4wm5"
server_key: "ede2717a5b4b22663545ae80b34c4a5b"
server_key_iv: "7ff47261cdf7d825017eaac1a37f6095"
server_key_version: "encryptionkey@0"
I20260925 09:20:25.902498 9097 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260925 09:20:25.906466 9112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:25.907979 9097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:20:25.908182 9097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "604d6f6b0eb14b33b2b95f1149dc1c4a"
format_stamp: "Formatted at 2026-09-25 09:20:25 on dist-test-slave-4wm5"
server_key: "ede2717a5b4b22663545ae80b34c4a5b"
server_key_iv: "7ff47261cdf7d825017eaac1a37f6095"
server_key_version: "encryptionkey@0"
I20260925 09:20:25.908411 9097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:25.920220 9097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:25.925371 9097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:25.925698 9097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:25.926805 9097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:25.928238 9097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:25.928344 9097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:25.928499 9097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:25.928570 9097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:25.964385 9097 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:42209
I20260925 09:20:25.964447 9224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:42209 every 8 connection(s)
I20260925 09:20:25.965720 9097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:25.969710 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9097
I20260925 09:20:25.969991 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:25.970817 3400 external_mini_cluster.cc:1428] Setting key c7c85b507161084c1f6f84aa99666071
Sep 25 09:20:25 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328025, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:25.991315 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:25.968424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48737 (local address 127.3.82.62:34015)
0925 09:20:25.968912 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:25.968918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:25.970329 (+ 1411us) server_negotiation.cc:407] Connection header received
0925 09:20:25.972775 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:25.972784 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:25.972885 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:25.972997 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:25.974825 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:25.976027 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:25.978161 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:25.978942 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:25.979098 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:25.984225 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:25.984233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:25.984239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:25.984286 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:25.987272 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:25.988461 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:25.988466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:25.988469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:25.988636 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:25.989248 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:25.989252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:25.989254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:25.989612 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:25.989742 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:25.990834 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0925 09:20:25.991056 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":229,"threads_started":1}
I20260925 09:20:25.993031 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:25.969110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34015 (local address 127.3.82.1:48737)
0925 09:20:25.970136 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:25.970191 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:25.972393 (+ 2202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:25.973308 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:25.973318 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:25.973984 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:25.974608 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:25.974626 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:25.976206 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:25.976210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:25.977985 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:25.978001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:25.979361 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:25.979371 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:25.979728 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:25.980606 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:25.980636 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:25.983922 (+ 3286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:25.987456 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:25.987469 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:25.987494 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:25.988197 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:25.988778 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:25.988786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:25.988793 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:25.989088 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:25.989763 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:25.989774 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:25.990324 (+ 550us) client_negotiation.cc:769] Sending connection context
0925 09:20:25.990668 (+ 344us) client_negotiation.cc:240] Negotiation successful
0925 09:20:25.991332 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":395,"threads_started":1}
I20260925 09:20:25.995040 9225 heartbeater.cc:347] Connected to a master server at 127.3.82.62:34015
I20260925 09:20:25.995400 9225 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:20:25.996223 9225 heartbeater.cc:511] Master 127.3.82.62:34015 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:20:25.998914 9036 ts_manager.cc:194] Registered new tserver with Master: 604d6f6b0eb14b33b2b95f1149dc1c4a (127.3.82.1:42209)
I20260925 09:20:26.001281 9036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:48737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:20:26.031556 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34015
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:26.401100 9231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:26.401484 9231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:26.401661 9231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:26.409502 9231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:26.409690 9231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:26.409909 9231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:26.420293 9231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:26.421749 9231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:26.423188 9231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:26.437873 9236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:26.437778 9237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:26.437753 9239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:26.438920 9231 server_base.cc:1034] running on GCE node
I20260925 09:20:26.439750 9231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:26.440815 9231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:26.442054 9231 hybrid_clock.cc:648] HybridClock initialized: now 1790328026441943 us; error 70 us; skew 500 ppm
I20260925 09:20:26.442448 9231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:26 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:26.446338 9231 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:20:26.448192 9231 webserver.cc:460] Webserver started at http://127.3.82.2:42013/ using document root <none> and password file <none>
I20260925 09:20:26.449013 9231 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:26.449183 9231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:26.449532 9231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:26.452894 9231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "cde44069cc494837b7505ff34cddce57"
format_stamp: "Formatted at 2026-09-25 09:20:26 on dist-test-slave-4wm5"
server_key: "bb152070d8071a9f1e18956ac387a971"
server_key_iv: "c4a21384d1eff4c37d981ed77cfc20cf"
server_key_version: "encryptionkey@0"
I20260925 09:20:26.453943 9231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "cde44069cc494837b7505ff34cddce57"
format_stamp: "Formatted at 2026-09-25 09:20:26 on dist-test-slave-4wm5"
server_key: "bb152070d8071a9f1e18956ac387a971"
server_key_iv: "c4a21384d1eff4c37d981ed77cfc20cf"
server_key_version: "encryptionkey@0"
I20260925 09:20:26.459897 9231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260925 09:20:26.463894 9246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:26.465515 9231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:20:26.465726 9231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "cde44069cc494837b7505ff34cddce57"
format_stamp: "Formatted at 2026-09-25 09:20:26 on dist-test-slave-4wm5"
server_key: "bb152070d8071a9f1e18956ac387a971"
server_key_iv: "c4a21384d1eff4c37d981ed77cfc20cf"
server_key_version: "encryptionkey@0"
I20260925 09:20:26.466040 9231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:26.497397 9231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:26.502311 9231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:26.502627 9231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:26.503676 9231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:26.506088 9231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:26.506193 9231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:26.506330 9231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:26.506413 9231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:26.542070 9231 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:35115
I20260925 09:20:26.542111 9358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:35115 every 8 connection(s)
I20260925 09:20:26.543380 9231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:26.546280 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9231
I20260925 09:20:26.546439 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:26.546967 3400 external_mini_cluster.cc:1428] Setting key 913f0a5af22d30b53432bf40e9ad835b
Sep 25 09:20:26 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:26.566291 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:26.546079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:56027 (local address 127.3.82.62:34015)
0925 09:20:26.546570 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:26.546575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:26.547938 (+ 1363us) server_negotiation.cc:407] Connection header received
0925 09:20:26.550298 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:26.550303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:26.550402 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:26.550521 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:26.552147 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:26.552893 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:26.554670 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:26.555111 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:26.555213 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:26.559619 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:26.559625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:26.559628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:26.559665 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:26.562473 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:26.563749 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:26.563766 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:26.563768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:26.563915 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:26.564393 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:26.564398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:26.564401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:26.564687 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:26.564850 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:26.565595 (+ 745us) server_negotiation.cc:299] Negotiation successful
0925 09:20:26.565868 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":257,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:20:26.567423 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:26.546732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34015 (local address 127.3.82.2:56027)
0925 09:20:26.547734 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:26.547786 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:26.550014 (+ 2228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:26.550785 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:26.550794 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:26.551336 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:26.551959 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:26.551978 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:26.553069 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:26.553073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:26.554452 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:26.554489 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:26.555357 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:26.555366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:26.555713 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:26.556500 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:26.556535 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:26.559334 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:26.562651 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:26.562662 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:26.562681 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:26.563594 (+ 913us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:26.564054 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:26.564059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:26.564063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:26.564272 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:26.564822 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:26.564830 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:26.565219 (+ 389us) client_negotiation.cc:769] Sending connection context
0925 09:20:26.565513 (+ 294us) client_negotiation.cc:240] Negotiation successful
0925 09:20:26.566156 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":506,"threads_started":1}
I20260925 09:20:26.569126 9359 heartbeater.cc:347] Connected to a master server at 127.3.82.62:34015
I20260925 09:20:26.569500 9359 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:20:26.570442 9359 heartbeater.cc:511] Master 127.3.82.62:34015 requested a full tablet report, sending...
I20260925 09:20:26.572100 9036 ts_manager.cc:194] Registered new tserver with Master: cde44069cc494837b7505ff34cddce57 (127.3.82.2:35115)
I20260925 09:20:26.572942 9036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:56027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:20:26.598685 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34015
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:26.945004 9365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:26.945333 9365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:26.945438 9365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:26.953642 9365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:26.953786 9365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:26.953989 9365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:26.964515 9365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:34015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:26.965965 9365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:26.967379 9365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:26.983069 9373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:26.983116 9371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:26.983069 9370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:26.984829 9365 server_base.cc:1034] running on GCE node
I20260925 09:20:26.985598 9365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:26.986796 9365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:26.988060 9365 hybrid_clock.cc:648] HybridClock initialized: now 1790328026987990 us; error 70 us; skew 500 ppm
I20260925 09:20:26.988435 9365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:26 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:26.991987 9365 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:20:26.993786 9365 webserver.cc:460] Webserver started at http://127.3.82.3:34293/ using document root <none> and password file <none>
I20260925 09:20:26.994699 9365 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:26.994858 9365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:26.995240 9365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:26.998649 9365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "3ab733ea249940c581fb4418cb962c65"
format_stamp: "Formatted at 2026-09-25 09:20:26 on dist-test-slave-4wm5"
server_key: "df0d54d855e44740b83e16d97e0df686"
server_key_iv: "27777100b8c8630add83c365cedabf3f"
server_key_version: "encryptionkey@0"
I20260925 09:20:26.999679 9365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "3ab733ea249940c581fb4418cb962c65"
format_stamp: "Formatted at 2026-09-25 09:20:26 on dist-test-slave-4wm5"
server_key: "df0d54d855e44740b83e16d97e0df686"
server_key_iv: "27777100b8c8630add83c365cedabf3f"
server_key_version: "encryptionkey@0"
I20260925 09:20:27.005673 9225 heartbeater.cc:503] Master 127.3.82.62:34015 was elected leader, sending a full tablet report...
I20260925 09:20:27.006076 9365 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:20:27.011009 9380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:27.012884 9365 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:20:27.013155 9365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "3ab733ea249940c581fb4418cb962c65"
format_stamp: "Formatted at 2026-09-25 09:20:26 on dist-test-slave-4wm5"
server_key: "df0d54d855e44740b83e16d97e0df686"
server_key_iv: "27777100b8c8630add83c365cedabf3f"
server_key_version: "encryptionkey@0"
I20260925 09:20:27.013437 9365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:27.046808 9365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:27.051558 9365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:27.051878 9365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:27.052947 9365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:27.054523 9365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:27.054639 9365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:27.054792 9365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:27.054876 9365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:27.091642 9365 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:42831
I20260925 09:20:27.091691 9492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:42831 every 8 connection(s)
I20260925 09:20:27.093037 9365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:27.101779 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9365
I20260925 09:20:27.102061 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:27.102614 3400 external_mini_cluster.cc:1428] Setting key f5277ef27fce6d6a92143cf35427dcac
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328026, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:27.116433 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.096203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:56525 (local address 127.3.82.62:34015)
0925 09:20:27.096762 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.096768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.097795 (+ 1027us) server_negotiation.cc:407] Connection header received
0925 09:20:27.099771 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.099776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.099870 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.099979 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.101710 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.102694 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.104667 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.105170 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.105273 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.109537 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.109542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.109546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.109585 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.112806 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.113884 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.113895 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.113898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.114052 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.114560 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.114563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.114565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.114867 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.114977 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.115816 (+ 839us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.116154 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":272,"threads_started":1}
I20260925 09:20:27.117807 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.096772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:34015 (local address 127.3.82.3:56525)
0925 09:20:27.097619 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:27.097661 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:27.099436 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:27.100227 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:27.100236 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:27.100850 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:27.101483 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:27.101501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.102856 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.102862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:27.104485 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:27.104502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.105425 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.105432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:27.105762 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.106689 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:27.106712 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:27.109181 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:27.113027 (+ 3846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:27.113035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:27.113050 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:27.113669 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:27.114211 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:27.114215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:27.114217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:27.114423 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:27.115006 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:27.115012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:27.115384 (+ 372us) client_negotiation.cc:769] Sending connection context
0925 09:20:27.115705 (+ 321us) client_negotiation.cc:240] Negotiation successful
0925 09:20:27.116270 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":312,"threads_started":1}
I20260925 09:20:27.119689 9493 heartbeater.cc:347] Connected to a master server at 127.3.82.62:34015
I20260925 09:20:27.120095 9493 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:27.120908 9493 heartbeater.cc:511] Master 127.3.82.62:34015 requested a full tablet report, sending...
I20260925 09:20:27.122629 9036 ts_manager.cc:194] Registered new tserver with Master: 3ab733ea249940c581fb4418cb962c65 (127.3.82.3:42831)
I20260925 09:20:27.123479 9036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:56525
I20260925 09:20:27.132550 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/test-admin.keytab.
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.164757 3400 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 09:20:27.171422 9501 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:27.167926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.3.82.1:42209)
0925 09:20:27.168640 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.168647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.168676 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:20:27.168744 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.168751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.169034 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.169181 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.170670 (+ 1489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39374: BlockingRecv error: recv got EOF from 127.0.0.1:39374 (error 108)
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":367,"threads_started":1}
W20260925 09:20:27.171417 3400 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc' of type FILE
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.173467 3400 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.175359 3400 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:27.193220 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.177586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.3.82.1:42209)
0925 09:20:27.177903 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.177909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.178048 (+ 139us) server_negotiation.cc:407] Connection header received
0925 09:20:27.178287 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.178297 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.178363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.178468 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.179571 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.180561 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.181877 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.182532 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.182683 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.186872 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.186888 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.186912 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.186965 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.190114 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.190953 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.190961 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.190963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.191104 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.191544 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.191549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.191553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.191965 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.192285 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.192644 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.192926 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260925 09:20:27.200364 9501 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:27.198087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.3.82.1:42209)
0925 09:20:27.198318 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.198325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.198579 (+ 254us) server_negotiation.cc:407] Connection header received
0925 09:20:27.198876 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.198881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.198978 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.199100 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.200062 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39392: BlockingRecv error: recv got EOF from 127.0.0.1:39392 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 09:20:27.200603 3400 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc' of type FILE
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.202593 3400 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.204439 3400 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:27.221549 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.206140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.3.82.1:42209)
0925 09:20:27.206373 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.206380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.206886 (+ 506us) server_negotiation.cc:407] Connection header received
0925 09:20:27.207000 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.207003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.207069 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.207158 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.208535 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.209299 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.210738 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.211186 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.211288 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.215606 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.215612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.215615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.215656 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.218568 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.219416 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.219421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.219423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.219561 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.219995 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.220000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.220003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.220415 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.220637 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.220952 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.221213 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260925 09:20:27.226769 9501 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:27.224511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.3.82.1:42209)
0925 09:20:27.224771 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.224787 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.225092 (+ 305us) server_negotiation.cc:407] Connection header received
0925 09:20:27.225294 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.225298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.225378 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.225500 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.226477 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39412: BlockingRecv error: recv got EOF from 127.0.0.1:39412 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 09:20:27.227005 3400 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790327973894689-3400-0/krb5kdc/krb5cc' of type FILE
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.229003 3400 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:27.230485 3400 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:27.246023 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.232209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.3.82.1:42209)
0925 09:20:27.232579 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.232585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.232697 (+ 112us) server_negotiation.cc:407] Connection header received
0925 09:20:27.232877 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.232882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.232950 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.233045 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.234201 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.234943 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.236363 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.236770 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.236870 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.240677 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.240683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.240686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.240726 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.243510 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.244179 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.244183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.244185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.244317 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.244640 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.244644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.244646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.244915 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.245148 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.245362 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.245552 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260925 09:20:27.248098 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9097
I20260925 09:20:27.262602 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.248878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.3.82.1:42209)
0925 09:20:27.249077 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.249081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.249552 (+ 471us) server_negotiation.cc:407] Connection header received
0925 09:20:27.249692 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.249695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.249755 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.249900 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.251033 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.251752 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.253065 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.253476 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.253577 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.256215 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.256221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.256224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.256266 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.259502 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.260417 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.260423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.260425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.260612 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.261043 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.261046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.261049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.261440 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.261657 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.262103 (+ 446us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.262308 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 09:20:27.263327 9220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:27.351052 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9097
I20260925 09:20:27.367274 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9231
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:20:27.385998 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.368224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.3.82.2:35115)
0925 09:20:27.368880 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.368888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.368914 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:20:27.368975 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.368981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.369245 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.369397 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.370650 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.371833 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.373220 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.373776 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.374101 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.378770 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.378797 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.378814 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.378860 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.382498 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.383548 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.383558 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.383560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.383712 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.384219 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.384223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.384225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.384631 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.384901 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.385235 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.385538 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":383,"threads_started":1}
I20260925 09:20:27.387385 9354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:27.472159 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9231
I20260925 09:20:27.488930 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9365
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[8991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:20:27.507603 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:27.490166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.3.82.3:42831)
0925 09:20:27.490861 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:27.490869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:27.490902 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:20:27.491004 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:27.491011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:27.491247 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:27.491555 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:27.492727 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.494009 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.495540 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:27.496136 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:27.496239 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:27.500967 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:27.500986 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:27.501002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:27.501046 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:27.504455 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.505274 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.505283 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.505286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.505426 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:27.505898 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:27.505903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:27.505905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:27.506296 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:27.506585 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:27.506973 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 09:20:27.507271 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":380,"threads_started":1}
I20260925 09:20:27.509202 9488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:27.594777 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9365
I20260925 09:20:27.610965 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9004
I20260925 09:20:27.612303 9066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:27.718889 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9004
2026-09-25T09:20:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4203 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[9521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[9521](info): set up 2 sockets
Sep 25 09:20:27 dist-test-slave-4wm5 krb5kdc[9521](info): commencing operation
krb5kdc: starting...
W20260925 09:20:28.818920 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:28 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab.
Entry for principal oryx/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab.
I20260925 09:20:29.058009 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36591
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab
--principal=oryx/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:29.403340 9534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:29.403733 9534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:29.403903 9534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:29.411441 9534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:29.411567 9534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:29.411659 9534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:29.411746 9534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:29.411861 9534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:29.422266 9534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab
--principal=oryx/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:29.423926 9534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:29.425343 9534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:29.439240 9539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:29.439198 9540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:29.439173 9542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:29.440637 9534 server_base.cc:1034] running on GCE node
I20260925 09:20:29.441591 9534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:29.443446 9534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:29.444761 9534 hybrid_clock.cc:648] HybridClock initialized: now 1790328029444707 us; error 51 us; skew 500 ppm
I20260925 09:20:29.445268 9534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:29 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328029, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:29.458406 9534 init.cc:377] Logged in from keytab as oryx/127.3.82.62@KRBTEST.COM (short username oryx)
I20260925 09:20:29.460477 9534 webserver.cc:460] Webserver started at http://127.3.82.62:33343/ using document root <none> and password file <none>
I20260925 09:20:29.461277 9534 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:29.461395 9534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:29.461795 9534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:29.465303 9534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "02ef09e5832b484f871ed069a66f2770"
format_stamp: "Formatted at 2026-09-25 09:20:29 on dist-test-slave-4wm5"
server_key: "cd71cfb501ff49bef866068b035786c1"
server_key_iv: "0b414886220e207eea407b45fcda92aa"
server_key_version: "encryptionkey@0"
I20260925 09:20:29.466252 9534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "02ef09e5832b484f871ed069a66f2770"
format_stamp: "Formatted at 2026-09-25 09:20:29 on dist-test-slave-4wm5"
server_key: "cd71cfb501ff49bef866068b035786c1"
server_key_iv: "0b414886220e207eea407b45fcda92aa"
server_key_version: "encryptionkey@0"
I20260925 09:20:29.472241 9534 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:20:29.476282 9549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:29.478257 9534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:29.478480 9534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "02ef09e5832b484f871ed069a66f2770"
format_stamp: "Formatted at 2026-09-25 09:20:29 on dist-test-slave-4wm5"
server_key: "cd71cfb501ff49bef866068b035786c1"
server_key_iv: "0b414886220e207eea407b45fcda92aa"
server_key_version: "encryptionkey@0"
I20260925 09:20:29.478690 9534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:29.506878 9534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:29.512192 9534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:29.512562 9534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:29.534102 9534 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36591
I20260925 09:20:29.534121 9600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36591 every 8 connection(s)
I20260925 09:20:29.535686 9534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:29.539628 9601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:29.542474 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9534
I20260925 09:20:29.542670 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:29.543467 3400 external_mini_cluster.cc:1428] Setting key e75be59f2bd56394d24c2ca1297daceb
I20260925 09:20:29.553424 9601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Bootstrap starting.
I20260925 09:20:29.557469 9601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:29.559154 9601 log.cc:824] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:20:29 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:29.562707 9601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: No bootstrap required, opened a new log
I20260925 09:20:29.570825 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:29.545982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.3.82.62:36591)
0925 09:20:29.546867 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:29.546880 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:29.546927 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:20:29.548778 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:29.548804 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:29.549270 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:29.549655 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:29.551659 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:29.553365 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:29.555250 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:29.556091 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:29.556242 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:29.561781 (+ 5539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:29.561790 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:29.561807 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:29.561924 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:29.565926 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:29.566790 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:29.566798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:29.566806 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:29.567002 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:29.567469 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:29.567477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:29.567479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:29.568122 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:29.568489 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:29.569044 (+ 555us) server_negotiation.cc:299] Negotiation successful
0925 09:20:29.569563 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":406,"threads_started":1}
I20260925 09:20:29.571480 9601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } }
I20260925 09:20:29.571926 9601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:29.572043 9601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ef09e5832b484f871ed069a66f2770, State: Initialized, Role: FOLLOWER
I20260925 09:20:29.572666 9601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } }
I20260925 09:20:29.572883 9601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:29.573077 9601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:29.573261 9601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:29.575793 9601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } }
I20260925 09:20:29.576320 9601 leader_election.cc:302] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02ef09e5832b484f871ed069a66f2770; no voters:
I20260925 09:20:29.576980 9601 leader_election.cc:288] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:29.577174 9606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:29.578037 9606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 LEADER]: Becoming Leader. State: Replica: 02ef09e5832b484f871ed069a66f2770, State: Running, Role: LEADER
I20260925 09:20:29.578787 9606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } }
I20260925 09:20:29.579278 9601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:29.582422 9607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } }
I20260925 09:20:29.582898 9607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:29.585016 9608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ef09e5832b484f871ed069a66f2770. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } }
I20260925 09:20:29.585466 9608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:29.586267 9613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:29.592715 9613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:29.604001 9613 catalog_manager.cc:1347] Generated new cluster ID: 15326be2d0954770a1c5730c683877ee
I20260925 09:20:29.604174 9613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:29.617322 9613 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:29.618883 9613 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:29.628844 9613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Generated new TSK 0
I20260925 09:20:29.629993 9613 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab.
Entry for principal oryx/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab.
I20260925 09:20:29.700313 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36591
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:30.062402 9627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:30.062708 9627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:30.062885 9627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:30.070724 9627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:30.070845 9627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:30.071023 9627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:30.081370 9627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:30.082887 9627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:30.084312 9627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:30.098903 9632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:30.098990 9633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:30.099198 9635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:30.100454 9627 server_base.cc:1034] running on GCE node
I20260925 09:20:30.101192 9627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:30.102417 9627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:30.103698 9627 hybrid_clock.cc:648] HybridClock initialized: now 1790328030103660 us; error 46 us; skew 500 ppm
I20260925 09:20:30.104167 9627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:30 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328030, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:30.108028 9627 init.cc:377] Logged in from keytab as oryx/127.3.82.1@KRBTEST.COM (short username oryx)
I20260925 09:20:30.109665 9627 webserver.cc:460] Webserver started at http://127.3.82.1:37255/ using document root <none> and password file <none>
I20260925 09:20:30.110450 9627 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:30.110550 9627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:30.110801 9627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:30.113790 9627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "a170fc17915b1b9b6e1080545a48a50c"
server_key_iv: "d127b3a98a0fe409a2eae8ecc1971ea5"
server_key_version: "encryptionkey@0"
I20260925 09:20:30.114722 9627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "a170fc17915b1b9b6e1080545a48a50c"
server_key_iv: "d127b3a98a0fe409a2eae8ecc1971ea5"
server_key_version: "encryptionkey@0"
I20260925 09:20:30.120507 9627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:20:30.124583 9642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:30.126524 9627 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:30.126725 9627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "a170fc17915b1b9b6e1080545a48a50c"
server_key_iv: "d127b3a98a0fe409a2eae8ecc1971ea5"
server_key_version: "encryptionkey@0"
I20260925 09:20:30.126933 9627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:30.145807 9627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:30.150906 9627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:30.151223 9627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:30.152343 9627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:30.153920 9627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:30.153999 9627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:30.154130 9627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:30.154193 9627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:30.192785 9627 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:36029
I20260925 09:20:30.192828 9754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:36029 every 8 connection(s)
I20260925 09:20:30.194187 9627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:30.196341 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9627
I20260925 09:20:30.196569 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:30.197446 3400 external_mini_cluster.cc:1428] Setting key 8b5ad63dbb7131b1443aaa7e70628f26
Sep 25 09:20:30 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328030, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:20:30.218206 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:30.197292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:40005 (local address 127.3.82.62:36591)
0925 09:20:30.197981 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:30.197988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:30.199373 (+ 1385us) server_negotiation.cc:407] Connection header received
0925 09:20:30.201438 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:30.201443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:30.201542 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:30.201667 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:30.203385 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.204114 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.205812 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.206293 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.206449 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:30.211086 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:30.211092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:30.211096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:30.211138 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:30.214329 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:30.215311 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:30.215322 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:30.215325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:30.215465 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:30.216027 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:30.216038 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:30.216040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:30.216420 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:30.216595 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:30.217489 (+ 894us) server_negotiation.cc:299] Negotiation successful
0925 09:20:30.217749 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":370,"threads_started":1}
I20260925 09:20:30.219458 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:30.198166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:40005)
0925 09:20:30.199189 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:30.199239 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:30.201136 (+ 1897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:30.202015 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:30.202023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:30.202598 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:30.203202 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:30.203220 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.204290 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.204295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:30.205643 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:30.205656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.206560 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.206567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:30.206920 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:30.207772 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:30.207833 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:30.210741 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:30.214546 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:30.214558 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:30.214579 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:30.215137 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:30.215613 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:30.215619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:30.215623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:30.215880 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:30.216586 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:30.216596 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:30.217026 (+ 430us) client_negotiation.cc:769] Sending connection context
0925 09:20:30.217385 (+ 359us) client_negotiation.cc:240] Negotiation successful
0925 09:20:30.218050 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":429,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.2@KRBTEST.COM" created.
I20260925 09:20:30.221663 9755 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36591
I20260925 09:20:30.222076 9755 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:30.223160 9755 heartbeater.cc:511] Master 127.3.82.62:36591 requested a full tablet report, sending...
I20260925 09:20:30.226037 9566 ts_manager.cc:194] Registered new tserver with Master: 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029)
I20260925 09:20:30.227607 9566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:40005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab.
Entry for principal oryx/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab.
I20260925 09:20:30.250844 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36591
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:30.600296 9761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:30.600606 9761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:30.600772 9761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:30.608418 9761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:30.608561 9761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:30.608754 9761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:30.619016 9761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:30.620491 9761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:30.621901 9761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:30.636247 9766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:30.636780 9767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:30.636758 9769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:30.637961 9761 server_base.cc:1034] running on GCE node
I20260925 09:20:30.638623 9761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:30.639652 9761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:30.640915 9761 hybrid_clock.cc:648] HybridClock initialized: now 1790328030640830 us; error 73 us; skew 500 ppm
I20260925 09:20:30.641255 9761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:30 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328030, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:30.644740 9761 init.cc:377] Logged in from keytab as oryx/127.3.82.2@KRBTEST.COM (short username oryx)
I20260925 09:20:30.646579 9761 webserver.cc:460] Webserver started at http://127.3.82.2:43679/ using document root <none> and password file <none>
I20260925 09:20:30.647303 9761 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:30.647423 9761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:30.647699 9761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:30.651015 9761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "2b460fc481fc69a0bd44523ce26cb10c"
server_key_iv: "d950cc98ebe7314b6e69763efc01b2df"
server_key_version: "encryptionkey@0"
I20260925 09:20:30.651837 9761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "2b460fc481fc69a0bd44523ce26cb10c"
server_key_iv: "d950cc98ebe7314b6e69763efc01b2df"
server_key_version: "encryptionkey@0"
I20260925 09:20:30.657785 9761 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:20:30.661917 9776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:30.663465 9761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:20:30.663681 9761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "2b460fc481fc69a0bd44523ce26cb10c"
server_key_iv: "d950cc98ebe7314b6e69763efc01b2df"
server_key_version: "encryptionkey@0"
I20260925 09:20:30.663944 9761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:30.681547 9761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:30.687150 9761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:30.687507 9761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:30.688694 9761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:30.690403 9761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:30.690518 9761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:30.690704 9761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:30.690804 9761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:30.728084 9761 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:44877
I20260925 09:20:30.728134 9888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:44877 every 8 connection(s)
I20260925 09:20:30.729478 9761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:30.734453 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9761
I20260925 09:20:30.734651 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:30.735276 3400 external_mini_cluster.cc:1428] Setting key 016c25eeabd6438a976e7816c8469b26
Sep 25 09:20:30 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328030, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:30.756714 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:30.732731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:55227 (local address 127.3.82.62:36591)
0925 09:20:30.733241 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:30.733247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:30.734814 (+ 1567us) server_negotiation.cc:407] Connection header received
0925 09:20:30.737188 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:30.737192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:30.737286 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:30.737418 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:30.739809 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.740590 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.743320 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.743797 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.743906 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:30.748419 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:30.748428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:30.748436 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:30.748487 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:30.752415 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:30.753380 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:30.753389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:30.753395 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:30.753598 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:30.754326 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:30.754334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:30.754341 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:30.754754 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:30.754918 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:30.755873 (+ 955us) server_negotiation.cc:299] Negotiation successful
0925 09:20:30.756175 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":266,"threads_started":1}
I20260925 09:20:30.757483 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:30.733403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:55227)
0925 09:20:30.734569 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:30.734638 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:30.736884 (+ 2246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:30.737726 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:30.737739 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:30.738649 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:30.739579 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:30.739612 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.740776 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.740784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:30.743128 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:30.743149 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:30.744054 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:30.744064 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:30.744433 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:30.745289 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:30.745313 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:30.748110 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:30.752626 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:30.752635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:30.752652 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:30.753200 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:30.753760 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:30.753765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:30.753768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:30.754163 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:30.754915 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:30.754923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:30.755429 (+ 506us) client_negotiation.cc:769] Sending connection context
0925 09:20:30.755712 (+ 283us) client_negotiation.cc:240] Negotiation successful
0925 09:20:30.756169 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":414,"threads_started":1}
I20260925 09:20:30.759266 9889 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36591
WARNING: no policy specified for oryx/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:20:30.759650 9889 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:30.760479 9889 heartbeater.cc:511] Master 127.3.82.62:36591 requested a full tablet report, sending...
I20260925 09:20:30.762461 9566 ts_manager.cc:194] Registered new tserver with Master: 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.3@KRBTEST.COM" created.
I20260925 09:20:30.763465 9566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:55227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab.
Entry for principal oryx/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab.
I20260925 09:20:30.795341 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36591
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:31.153227 9895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:31.153601 9895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:31.153771 9895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:31.161962 9895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:31.162150 9895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:31.162410 9895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:31.173404 9895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:31.174935 9895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:31.176424 9895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:31.192212 9901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:31.192637 9903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:31.193634 9895 server_base.cc:1034] running on GCE node
W20260925 09:20:31.197472 9900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:31.198349 9895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:31.199515 9895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:31.200748 9895 hybrid_clock.cc:648] HybridClock initialized: now 1790328031200676 us; error 73 us; skew 500 ppm
I20260925 09:20:31.201123 9895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:31 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328031, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:31.204993 9895 init.cc:377] Logged in from keytab as oryx/127.3.82.3@KRBTEST.COM (short username oryx)
I20260925 09:20:31.206795 9895 webserver.cc:460] Webserver started at http://127.3.82.3:37841/ using document root <none> and password file <none>
I20260925 09:20:31.207618 9895 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:31.207782 9895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:31.208226 9895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:31.211647 9895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "86845cbeecfb4beca94170ae87538a59"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "8074407ad92a52fd2743d25afcbbadab"
server_key_iv: "ad088bb5e7f429427e43508c7cf80eb5"
server_key_version: "encryptionkey@0"
I20260925 09:20:31.212553 9895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "86845cbeecfb4beca94170ae87538a59"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "8074407ad92a52fd2743d25afcbbadab"
server_key_iv: "ad088bb5e7f429427e43508c7cf80eb5"
server_key_version: "encryptionkey@0"
I20260925 09:20:31.218709 9895 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:20:31.222982 9910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:31.224689 9895 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:20:31.224927 9895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "86845cbeecfb4beca94170ae87538a59"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "8074407ad92a52fd2743d25afcbbadab"
server_key_iv: "ad088bb5e7f429427e43508c7cf80eb5"
server_key_version: "encryptionkey@0"
I20260925 09:20:31.225168 9895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:31.231750 9755 heartbeater.cc:503] Master 127.3.82.62:36591 was elected leader, sending a full tablet report...
I20260925 09:20:31.238603 9895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:31.243600 9895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:31.243937 9895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:31.245014 9895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:31.246551 9895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:31.246655 9895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:31.246805 9895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:31.246879 9895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:31.283551 9895 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:39133
I20260925 09:20:31.283563 10022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:39133 every 8 connection(s)
I20260925 09:20:31.284960 9895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:31.289853 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 9895
I20260925 09:20:31.290050 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:31.290652 3400 external_mini_cluster.cc:1428] Setting key aa5e6a50f30078d70d69f870d6918781
Sep 25 09:20:31 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328031, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:31.311499 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.288160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50491 (local address 127.3.82.62:36591)
0925 09:20:31.288695 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:31.288700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:31.289924 (+ 1224us) server_negotiation.cc:407] Connection header received
0925 09:20:31.292513 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:31.292518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:31.292642 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:31.292770 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:31.295244 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.296644 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.298977 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.299543 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.299701 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.303854 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:31.303861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:31.303864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:31.303911 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:31.307580 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.308561 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.308569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.308579 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.308722 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.309298 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.309302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.309304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.309605 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:31.309813 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:31.310765 (+ 952us) server_negotiation.cc:299] Negotiation successful
0925 09:20:31.311071 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":253,"threads_started":1}
I20260925 09:20:31.312690 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.288756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:50491)
0925 09:20:31.289679 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:31.289738 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:31.292192 (+ 2454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:31.293460 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:31.293473 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:31.294164 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:31.294972 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:31.295005 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.296841 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.296847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:31.298771 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:31.298788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.299857 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.299863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:31.300118 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.300871 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:31.300911 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:31.303534 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:31.307795 (+ 4261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:31.307815 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:31.307833 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:31.308385 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:31.308857 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:31.308863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:31.308867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:31.309107 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:31.309773 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:31.309784 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:31.310268 (+ 484us) client_negotiation.cc:769] Sending connection context
0925 09:20:31.310640 (+ 372us) client_negotiation.cc:240] Negotiation successful
0925 09:20:31.311318 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":310,"threads_started":1}
I20260925 09:20:31.314525 10023 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36591
I20260925 09:20:31.314930 10023 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:31.315745 10023 heartbeater.cc:511] Master 127.3.82.62:36591 requested a full tablet report, sending...
I20260925 09:20:31.317646 9566 ts_manager.cc:194] Registered new tserver with Master: 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133)
I20260925 09:20:31.318584 9566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:50491
I20260925 09:20:31.320748 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.82.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.61.keytab.
Entry for principal oryx/127.3.82.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.61.keytab.
I20260925 09:20:31.378420 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:41277
--webserver_interface=127.3.82.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.61.keytab
--principal=oryx/127.3.82.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36591,127.3.82.61:41277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:31.735113 10029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:31.735476 10029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:31.735661 10029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:31.743952 10029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:31.744113 10029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:31.744226 10029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:31.744330 10029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:31.744414 10029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:31.755357 10029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36591,127.3.82.61:41277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.61.keytab
--principal=oryx/127.3.82.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:41277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.82.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
W20260925 09:20:31.755808 10029 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.82.62:36591,127.3.82.61:41277'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 09:20:31.756927 10029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:31.758391 10029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:20:31.767203 9889 heartbeater.cc:503] Master 127.3.82.62:36591 was elected leader, sending a full tablet report...
W20260925 09:20:31.771684 10035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:31.771733 10034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:31.771744 10037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:31.772576 10029 server_base.cc:1034] running on GCE node
I20260925 09:20:31.773509 10029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:31.775164 10029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:31.776484 10029 hybrid_clock.cc:648] HybridClock initialized: now 1790328031776396 us; error 62 us; skew 500 ppm
I20260925 09:20:31.776873 10029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:31 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328031, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:31.780573 10029 init.cc:377] Logged in from keytab as oryx/127.3.82.61@KRBTEST.COM (short username oryx)
I20260925 09:20:31.782442 10029 webserver.cc:460] Webserver started at http://127.3.82.61:33035/ using document root <none> and password file <none>
I20260925 09:20:31.783250 10029 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:31.783390 10029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:31.783708 10029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:31.787177 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/instance:
uuid: "14d61c1451e84b708de00695bfe689f5"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "bc3ffedfda711bb9dd3aec96aca05579"
server_key_iv: "8d0bf975522ac0152174bc2794191cca"
server_key_version: "encryptionkey@0"
I20260925 09:20:31.787986 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal/instance:
uuid: "14d61c1451e84b708de00695bfe689f5"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "bc3ffedfda711bb9dd3aec96aca05579"
server_key_iv: "8d0bf975522ac0152174bc2794191cca"
server_key_version: "encryptionkey@0"
I20260925 09:20:31.794225 10029 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:20:31.798120 10044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:31.799760 10029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:20:31.799962 10029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
uuid: "14d61c1451e84b708de00695bfe689f5"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "bc3ffedfda711bb9dd3aec96aca05579"
server_key_iv: "8d0bf975522ac0152174bc2794191cca"
server_key_version: "encryptionkey@0"
I20260925 09:20:31.800174 10029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:31.894726 10029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:31.899487 10029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:31.899717 10029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:31.920095 10029 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.61:41277
I20260925 09:20:31.920114 10095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.61:41277 every 8 connection(s)
I20260925 09:20:31.921624 10029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
I20260925 09:20:31.923519 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 10029
I20260925 09:20:31.926190 10096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:31.935629 10096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:20:31 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
Sep 25 09:20:31 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328031, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.61@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:31.949115 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.925890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41814 (local address 127.3.82.61:41277)
0925 09:20:31.926745 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:31.926757 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:31.926804 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:20:31.928482 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:31.928505 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:31.929017 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:31.929431 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:31.931407 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.932995 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.934975 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.935765 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.935948 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.940834 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:31.940849 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:31.940865 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:31.940945 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:31.944266 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.945054 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.945062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.945069 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.945252 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.945793 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.945797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.945799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.946495 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:31.946738 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:31.947378 (+ 640us) server_negotiation.cc:299] Negotiation successful
0925 09:20:31.947833 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":403,"threads_started":1}
I20260925 09:20:31.955343 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.938667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.3.82.62:36591)
0925 09:20:31.939195 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:31.939204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:31.939998 (+ 794us) server_negotiation.cc:407] Connection header received
0925 09:20:31.940086 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:31.940090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:31.940170 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:31.940297 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:31.941973 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.942737 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.944239 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.944661 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.944811 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.948648 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:31.948655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:31.948659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:31.948710 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:31.952271 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.953122 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.953127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.953130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.953273 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.953723 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.953727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.953730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.954210 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:31.954342 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:31.954887 (+ 545us) server_negotiation.cc:299] Negotiation successful
0925 09:20:31.955091 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
I20260925 09:20:31.955360 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.938658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:58632)
0925 09:20:31.939658 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:31.939708 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:31.939880 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:31.940390 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:31.940398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:31.941408 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:31.941766 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:31.941776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.942893 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.942899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:31.944076 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:31.944088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.944848 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.944854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:31.945051 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.945686 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:31.945715 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:31.948349 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:31.952437 (+ 4088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:31.952447 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:31.952455 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:31.952970 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:31.953419 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:31.953423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:31.953426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:31.953571 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:31.954354 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:31.954361 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:31.954580 (+ 219us) client_negotiation.cc:769] Sending connection context
0925 09:20:31.954758 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 09:20:31.955037 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":569,"threads_started":1}
I20260925 09:20:31.957229 10096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:20:31 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328031, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.61@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
I20260925 09:20:31.971638 10101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.958160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41818)
0925 09:20:31.958355 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:31.958375 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:31.958505 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:31.958847 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:31.958851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:31.959237 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:31.959372 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:31.959380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.960348 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.960356 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:31.961223 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:31.961234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.962009 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.962015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:31.962199 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.962781 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:31.962798 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:31.965555 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:31.969052 (+ 3497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:31.969056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:31.969059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:31.969593 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:31.970103 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:31.970106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:31.970109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:31.970245 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:31.970865 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:31.970869 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:31.971014 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 09:20:31.971152 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 09:20:31.971379 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 09:20:31.971946 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:31.958142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.3.82.61:41277)
0925 09:20:31.958351 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:31.958357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:31.958444 (+ 87us) server_negotiation.cc:407] Connection header received
0925 09:20:31.958670 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:31.958675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:31.958734 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:31.958830 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:31.959500 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.960227 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.961362 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:31.961785 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:31.961976 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:31.965770 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:31.965775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:31.965779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:31.965809 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:31.968879 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.969720 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.969729 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.969731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.969978 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:31.970356 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:31.970360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:31.970362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:31.970744 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:31.970861 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:31.971260 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 09:20:31.971594 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":67}
I20260925 09:20:31.976948 10096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Bootstrap starting.
I20260925 09:20:31.980913 10096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:31.982117 10096 log.cc:824] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:31.985452 10096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: No bootstrap required, opened a new log
I20260925 09:20:31.993014 10096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } }
I20260925 09:20:31.993403 10096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:31.993516 10096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d61c1451e84b708de00695bfe689f5, State: Initialized, Role: FOLLOWER
I20260925 09:20:31.994251 10096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } }
I20260925 09:20:31.995150 10103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } } }
I20260925 09:20:31.995528 10103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:20:31.996274 10096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:20:32.007102 10114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:20:32.007382 10114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:20:32.009868 10029 master_runner.cc:428] Detected that this master 14d61c1451e84b708de00695bfe689f5 is joining an existing cluster
I20260925 09:20:32.009994 10029 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.82.61:41277
I20260925 09:20:32.037308 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.015050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.3.82.62:36591)
0925 09:20:32.015242 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.015251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.020689 (+ 5438us) server_negotiation.cc:407] Connection header received
0925 09:20:32.020920 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.020927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.021028 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.021130 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.022721 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.023833 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.025989 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.026452 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.026597 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.029246 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.029255 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.029262 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.029311 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.033628 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.034626 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.034635 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.034639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.034807 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.035307 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.035312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.035316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.035713 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.035859 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.036517 (+ 658us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.036931 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:20:32.039093 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.014465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:58636)
0925 09:20:32.020525 (+ 6060us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.020551 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.020765 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.021341 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.021347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.022081 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.022527 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.022542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.024072 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.024078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.025767 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.025783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.026665 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.026672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.026868 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.027684 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.027708 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.029001 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.033865 (+ 4864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.033871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.033875 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.034460 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.034980 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.034985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.034988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.035164 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.035954 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.035959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.036185 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.037588 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.038514 (+ 926us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5892,"thread_start_us":12582,"threads_started":1}
WARNING: no policy specified for oryx/127.3.82.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.60@KRBTEST.COM" created.
I20260925 09:20:32.048339 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.014489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41830)
0925 09:20:32.027449 (+ 12960us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.027474 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.027668 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.028219 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.028226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.029001 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.029360 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.029373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.033559 (+ 4186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.033568 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.035492 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.035513 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.036934 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.036942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.037153 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.038470 (+ 1317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.038493 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.039643 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.044315 (+ 4672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.044321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.044325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.044918 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.045574 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.045583 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.045591 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.045804 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.046850 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.046858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.047061 (+ 203us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.047249 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.047655 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12782,"mutex_wait_us":1,"thread_start_us":5678,"threads_started":1}
I20260925 09:20:32.049526 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.014988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.3.82.61:41277)
0925 09:20:32.015191 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.015198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.027563 (+ 12365us) server_negotiation.cc:407] Connection header received
0925 09:20:32.027816 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.027823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.027914 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.027997 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.029514 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.030867 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.035665 (+ 4798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.036463 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.036581 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.039894 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.039904 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.039912 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.039950 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.044096 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.045075 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.045080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.045084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.045254 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.046032 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.046041 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.046048 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.046583 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.046788 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.048714 (+ 1926us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.049018 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328029, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.62@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
I20260925 09:20:32.074659 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.047888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.3.82.61:41277)
0925 09:20:32.049796 (+ 1908us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.049801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.053893 (+ 4092us) server_negotiation.cc:407] Connection header received
0925 09:20:32.054024 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.054034 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.054168 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.054316 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.056273 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.057505 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.060091 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.060852 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.061014 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.066335 (+ 5321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.066347 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.066357 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.066419 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.070977 (+ 4558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.072127 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.072132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.072135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.072296 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.072880 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.072889 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.072895 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.073272 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.073421 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.074109 (+ 688us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.074313 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1788,"thread_start_us":6317,"threads_started":1}
I20260925 09:20:32.074838 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.047508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41832)
0925 09:20:32.053266 (+ 5758us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.053334 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.053618 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.054549 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.054562 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.055667 (+ 1105us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.056074 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.056092 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.057709 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.057719 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.059834 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.059858 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.061102 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.061112 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.061418 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.062455 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.062488 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.065977 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.071183 (+ 5206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.071203 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.071249 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.071972 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.072426 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.072431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.072435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.072711 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.073418 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.073428 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.073710 (+ 282us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.073970 (+ 260us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.074373 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5416,"thread_start_us":5206,"threads_started":1}
I20260925 09:20:32.077113 9566 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.82.61:41277: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "14d61c1451e84b708de00695bfe689f5"
member_type: NON_VOTER
last_known_addr {
host: "127.3.82.61"
port: 41277
}
attrs {
promote: true
}
}
dest_uuid: "02ef09e5832b484f871ed069a66f2770"
cas_config_opid_index: -1
I20260925 09:20:32.078461 9566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.60.keytab.
Entry for principal oryx/127.3.82.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.60.keytab.
I20260925 09:20:32.082818 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:45463
--webserver_interface=127.3.82.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.60.keytab
--principal=oryx/127.3.82.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:20:32.083285 10071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:32.087265 10071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Refusing update from remote peer 02ef09e5832b484f871ed069a66f2770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 09:20:32.090508 10129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 14d61c1451e84b708de00695bfe689f5 (127.3.82.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.092360 10128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:20:32.104529 10129 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.82.61:41277
I20260925 09:20:32.106806 10029 master.cc:537] Master@127.3.82.61:41277 shutting down...
I20260925 09:20:32.108210 10128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.108600 10128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:32.128083 10071 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 14d61c1451e84b708de00695bfe689f5 (127.3.82.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.146171 10138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.146692 10138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:20:32.146725 10134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.147166 10134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:20:32.147162 10103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ef09e5832b484f871ed069a66f2770. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.147533 10103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:20:32.150472 10029 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 LEARNER]: Raft consensus shutting down.
I20260925 09:20:32.151084 10029 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 LEARNER]: Raft consensus is shut down!
I20260925 09:20:32.151273 10029 tablet_replica.cc:330] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: stopping tablet replica
I20260925 09:20:32.160632 10029 master.cc:552] Master@127.3.82.61:41277 shutdown complete.
I20260925 09:20:32.160743 10029 master_runner.cc:305] Clearing existing system tablet
I20260925 09:20:32.161234 10029 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:32.161345 10029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:32.164947 10029 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260925 09:20:32.167217 10140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:32.167732 10029 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:32.168043 10029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 09:20:32.168212 10029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
uuid: "14d61c1451e84b708de00695bfe689f5"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "bc3ffedfda711bb9dd3aec96aca05579"
server_key_iv: "8d0bf975522ac0152174bc2794191cca"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.173091 10029 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:32.180593 10029 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 09:20:32.180833 10029 log.cc:1196] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 09:20:32.181422 10029 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Deleting consensus metadata
I20260925 09:20:32.181722 10029 master_runner.cc:315] Copying system tablet from 127.3.82.62:36591
I20260925 09:20:32.187831 10029 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: tablet copy: Beginning tablet copy session from remote peer at address 127.3.82.62:36591
I20260925 09:20:32.214438 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.188778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58652 (local address 127.3.82.62:36591)
0925 09:20:32.188985 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.188992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.198611 (+ 9619us) server_negotiation.cc:407] Connection header received
0925 09:20:32.198834 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.198841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.198968 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.199111 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.200291 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.201480 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.203500 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.204178 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.204319 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.206956 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.206965 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.206972 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.207027 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.210949 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.211965 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.211971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.211975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.212141 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.212647 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.212652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.212655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.213036 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.213189 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.213745 (+ 556us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.214047 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:20:32.214550 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.188721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:58652)
0925 09:20:32.198431 (+ 9710us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.198465 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.198674 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.199160 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.199166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.199734 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.200074 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.200089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.201670 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.201677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.203308 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.203328 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.204371 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.204378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.204623 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.205429 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.205457 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.206660 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.211123 (+ 4463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.211131 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.211138 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.211790 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.212279 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.212285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.212290 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.212502 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.213171 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.213177 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.213394 (+ 217us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.213586 (+ 192us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.213984 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9517,"thread_start_us":9315,"threads_started":1}
I20260925 09:20:32.215879 9586 tablet_copy_service.cc:140] P 02ef09e5832b484f871ed069a66f2770: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 14d61c1451e84b708de00695bfe689f5 ({username='oryx', principal='oryx/127.3.82.61@KRBTEST.COM'} at 127.0.0.1:58652)
I20260925 09:20:32.216257 9586 tablet_copy_service.cc:161] P 02ef09e5832b484f871ed069a66f2770: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 14d61c1451e84b708de00695bfe689f5 at {username='oryx', principal='oryx/127.3.82.61@KRBTEST.COM'} at 127.0.0.1:58652: session id = 14d61c1451e84b708de00695bfe689f5-00000000000000000000000000000000
I20260925 09:20:32.220914 9586 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 09:20:32.223641 10029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 09:20:32.231992 10029 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: tablet copy: Starting download of 0 data blocks...
I20260925 09:20:32.232405 10029 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: tablet copy: Starting download of 1 WAL segments...
I20260925 09:20:32.236220 10029 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 09:20:32.241326 9586 tablet_copy_service.cc:342] P 02ef09e5832b484f871ed069a66f2770: Request end of tablet copy session 14d61c1451e84b708de00695bfe689f5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.82.61@KRBTEST.COM'} at 127.0.0.1:58652
I20260925 09:20:32.241600 9586 tablet_copy_service.cc:433] P 02ef09e5832b484f871ed069a66f2770: ending tablet copy session 14d61c1451e84b708de00695bfe689f5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 14d61c1451e84b708de00695bfe689f5
I20260925 09:20:32.254419 10029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:32.257333 10147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:32.259122 10148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:32.265901 10150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:32.266280 10029 server_base.cc:1034] running on GCE node
I20260925 09:20:32.266995 10029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:32.267784 10029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:32.269251 10029 hybrid_clock.cc:648] HybridClock initialized: now 1790328032269168 us; error 47 us; skew 500 ppm
I20260925 09:20:32.269572 10029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328032, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:32.272225 10029 init.cc:377] Logged in from keytab as oryx/127.3.82.61@KRBTEST.COM (short username oryx)
I20260925 09:20:32.273573 10029 webserver.cc:460] Webserver started at http://127.3.82.61:35681/ using document root <none> and password file <none>
I20260925 09:20:32.274004 10029 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:32.274096 10029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:32.277562 10029 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260925 09:20:32.279912 10157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:32.280391 10029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 09:20:32.280527 10029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
uuid: "14d61c1451e84b708de00695bfe689f5"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "bc3ffedfda711bb9dd3aec96aca05579"
server_key_iv: "8d0bf975522ac0152174bc2794191cca"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.280723 10029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:32.322082 10023 heartbeater.cc:503] Master 127.3.82.62:36591 was elected leader, sending a full tablet report...
I20260925 09:20:32.338634 10029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:32.342952 10029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:32.378861 10029 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.61:41277
I20260925 09:20:32.379123 10208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.61:41277 every 8 connection(s)
I20260925 09:20:32.380160 10029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
I20260925 09:20:32.386603 10209 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:20:32.391523 10209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Bootstrap starting.
I20260925 09:20:32.408495 10209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:20:32.409121 10209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: Bootstrap complete.
I20260925 09:20:32.410487 10209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } }
I20260925 09:20:32.410794 10209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 14d61c1451e84b708de00695bfe689f5, State: Initialized, Role: LEARNER
I20260925 09:20:32.411108 10209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } }
I20260925 09:20:32.412032 10212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.412380 10209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:32.412506 10212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:20:32.421973 10223 catalog_manager.cc:1259] Loaded cluster ID: 15326be2d0954770a1c5730c683877ee
I20260925 09:20:32.422089 10223 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: loading cluster ID for follower catalog manager: success
I20260925 09:20:32.425972 10223 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: acquiring CA information for follower catalog manager: success
I20260925 09:20:32.428738 10223 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:20:32.507706 10130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:32.508098 10130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:32.508253 10130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:32.516400 10130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:32.516598 10130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:32.516705 10130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:32.516809 10130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:32.516888 10130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:32.527774 10130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.60.keytab
--principal=oryx/127.3.82.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:45463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.82.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:32.529397 10130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:32.531052 10130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:32.550040 10228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:32.550014 10229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:32.557170 10231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:32.559103 10130 server_base.cc:1034] running on GCE node
I20260925 09:20:32.560180 10130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:32.562175 10130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:32.563441 10130 hybrid_clock.cc:648] HybridClock initialized: now 1790328032563389 us; error 42 us; skew 500 ppm
I20260925 09:20:32.563876 10130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:20:32.564397 10129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.564695 10129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328032, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:32.568262 10130 init.cc:377] Logged in from keytab as oryx/127.3.82.60@KRBTEST.COM (short username oryx)
I20260925 09:20:32.570380 10130 webserver.cc:460] Webserver started at http://127.3.82.60:38795/ using document root <none> and password file <none>
I20260925 09:20:32.571292 10130 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:32.571429 10130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:32.571738 10130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:32.572979 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.565637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.3.82.61:41277)
0925 09:20:32.566307 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.566311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.566327 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:20:32.566390 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.566393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.566463 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.566622 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:32.567056 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.568442 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.570932 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.571958 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.572070 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.572310 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.572590 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.572707 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":378,"threads_started":1}
I20260925 09:20:32.572985 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.564816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41838)
0925 09:20:32.565175 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.565197 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.565331 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.566665 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.566670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.566702 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:32.566896 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.566905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.568668 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.568675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.570724 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.570744 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.572126 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.572132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.572358 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.572423 (+ 65us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.572540 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.572716 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260925 09:20:32.574190 10212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ef09e5832b484f871ed069a66f2770. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: NON_VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: true } } }
I20260925 09:20:32.574448 10212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: LEARNER
I20260925 09:20:32.575204 10127 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: attempting to promote NON_VOTER 14d61c1451e84b708de00695bfe689f5 to VOTER
I20260925 09:20:32.576009 10130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/instance:
uuid: "c429f23af11e4acba958cf1dcde71bef"
format_stamp: "Formatted at 2026-09-25 09:20:32 on dist-test-slave-4wm5"
server_key: "bdcbc98571a870a68b3d551a1103e9d9"
server_key_iv: "6a7432bcfd748b4a82cf9fe2b0458727"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.576215 10127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } }
I20260925 09:20:32.577126 10130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal/instance:
uuid: "c429f23af11e4acba958cf1dcde71bef"
format_stamp: "Formatted at 2026-09-25 09:20:32 on dist-test-slave-4wm5"
server_key: "bdcbc98571a870a68b3d551a1103e9d9"
server_key_iv: "6a7432bcfd748b4a82cf9fe2b0458727"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.578351 10184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 LEARNER]: Refusing update from remote peer 02ef09e5832b484f871ed069a66f2770: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 09:20:32.579074 10129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:20:32.582702 10127 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 14d61c1451e84b708de00695bfe689f5 (127.3.82.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } }
I20260925 09:20:32.583608 10184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 14d61c1451e84b708de00695bfe689f5 (127.3.82.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } }
I20260925 09:20:32.584968 10130 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260925 09:20:32.586560 10212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } }
I20260925 09:20:32.586807 10212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:20:32.586731 10129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } }
I20260925 09:20:32.586822 10128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } }
I20260925 09:20:32.587040 10129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:32.587080 10128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:32.589797 10240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:32.591509 10130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:32.591835 10130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
uuid: "c429f23af11e4acba958cf1dcde71bef"
format_stamp: "Formatted at 2026-09-25 09:20:32 on dist-test-slave-4wm5"
server_key: "bdcbc98571a870a68b3d551a1103e9d9"
server_key_iv: "6a7432bcfd748b4a82cf9fe2b0458727"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.592028 10130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:32.621693 10130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:32.626538 10130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:32.626860 10130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:32.646804 10130 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.60:45463
I20260925 09:20:32.646883 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.60:45463 every 8 connection(s)
I20260925 09:20:32.648283 10130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
I20260925 09:20:32.652420 10292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:32.654341 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 10130
I20260925 09:20:32.662587 10292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.60@KRBTEST.COM
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328032, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.60@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:32.680390 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.656987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.3.82.60:45463)
0925 09:20:32.658060 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.658074 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.658121 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:20:32.659758 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.659782 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.660278 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.660740 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.662404 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.663961 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.665552 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.666575 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.667083 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.671936 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.671947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.671964 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.672020 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.675526 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.676272 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.676285 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.676297 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.676478 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.676885 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.676893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.676896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.677448 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.677734 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.678332 (+ 598us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.678826 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":516,"threads_started":1}
I20260925 09:20:32.682804 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.665600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:58654)
0925 09:20:32.666593 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.666634 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.666845 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.667373 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.667385 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.668223 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.668550 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.668561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.669710 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.669716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.670988 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.671005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.671978 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.671985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.672135 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.672961 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.672998 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.675566 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.679271 (+ 3705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.679290 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.679296 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.679893 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.680342 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.680349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.680352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.680660 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.681634 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.681641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.681894 (+ 253us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.682095 (+ 201us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.682396 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":570,"threads_started":1}
I20260925 09:20:32.682896 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.665740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.3.82.62:36591)
0925 09:20:32.665990 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.665996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.666721 (+ 725us) server_negotiation.cc:407] Connection header received
0925 09:20:32.667057 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.667061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.667125 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.667241 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.668701 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.669559 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.671157 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.671665 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.671839 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.675801 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.675806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.675809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.675846 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.679102 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.680038 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.680044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.680047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.680211 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.680851 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.680856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.680859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.681275 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.681548 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.682289 (+ 741us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.682541 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260925 09:20:32.684916 10292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328032, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.60@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
I20260925 09:20:32.699308 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.685621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41852)
0925 09:20:32.685816 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.685866 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.685970 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.686285 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.686289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.686604 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.686735 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.686743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.687860 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.687865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.688837 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.688850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.689590 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.689595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.689716 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.690426 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.690450 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.692891 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.696570 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.696575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.696578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.697067 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.697603 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.697606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.697609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.697768 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.698529 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.698535 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.698703 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.698830 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.699080 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 09:20:32.699371 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.685756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.3.82.61:41277)
0925 09:20:32.686020 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.686024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.686041 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:32.686104 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.686108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.686168 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.686248 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.686903 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.687709 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.689014 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.689433 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.689540 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.693126 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.693131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.693134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.693171 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.696340 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.697258 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.697262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.697264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.697432 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.697990 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.697995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.697997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.698308 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.698502 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.698941 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.699136 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260925 09:20:32.700702 10292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328032, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.60@KRBTEST.COM for oryx/127.3.82.60@KRBTEST.COM
I20260925 09:20:32.717062 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.703258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.3.82.60:45463)
0925 09:20:32.703437 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.703445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.703562 (+ 117us) server_negotiation.cc:407] Connection header received
0925 09:20:32.703763 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.703768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.703840 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.703955 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.704720 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.705456 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.706740 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.707202 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.707371 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.711032 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.711043 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.711046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.711077 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.714079 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.714883 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.714891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.714897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.715036 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.715430 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.715437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.715440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.715744 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.715904 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.716420 (+ 516us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.716718 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:20:32.717084 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.703102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.0.0.1:48410)
0925 09:20:32.703437 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.703462 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.703642 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.703972 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.703977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.704425 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.704572 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.704582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.705609 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.705615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.706585 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.706598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.707402 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.707409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.707620 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.708340 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.708358 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.710808 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.714286 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.714291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.714294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.714735 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.715161 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.715167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.715171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.715321 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.715864 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.715868 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.716021 (+ 153us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.716304 (+ 283us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.716601 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260925 09:20:32.723263 10292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Bootstrap starting.
I20260925 09:20:32.726725 10292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:32.727906 10292 log.cc:824] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:32.731421 10292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: No bootstrap required, opened a new log
I20260925 09:20:32.740037 10292 raft_consensus.cc:348] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } }
I20260925 09:20:32.740406 10292 raft_consensus.cc:374] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:32.740497 10292 raft_consensus.cc:729] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c429f23af11e4acba958cf1dcde71bef, State: Initialized, Role: FOLLOWER
I20260925 09:20:32.741065 10292 consensus_queue.cc:260] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } }
I20260925 09:20:32.742010 10298 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } } }
I20260925 09:20:32.742406 10298 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:20:32.743268 10292 sys_catalog.cc:564] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:20:32.753095 10309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:20:32.753291 10309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:20:32.758322 10130 master_runner.cc:428] Detected that this master c429f23af11e4acba958cf1dcde71bef is joining an existing cluster
I20260925 09:20:32.758450 10130 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.82.60:45463
I20260925 09:20:32.781946 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.762729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.3.82.62:36591)
0925 09:20:32.762964 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.762971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.763700 (+ 729us) server_negotiation.cc:407] Connection header received
0925 09:20:32.763965 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.763974 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.764084 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.764179 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.766232 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.767706 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.770118 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.770890 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.771009 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.773806 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.773813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.773818 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.773888 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.777888 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.778869 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.778875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.778878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.779051 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.779595 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.779603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.779608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.780020 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.780160 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.781167 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.781459 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:20:32.783288 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.762787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:58666)
0925 09:20:32.763577 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.763592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.763770 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.764476 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.764488 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.765464 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.765957 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.765975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.767982 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.767990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.769941 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.769956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.771326 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.771333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.771520 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.772299 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.772320 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.773571 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.778158 (+ 4587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.778166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.778170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.778709 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.779202 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.779206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.779210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.779423 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.780260 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.780267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.780810 (+ 543us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.782221 (+ 1411us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.782651 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":77,"thread_start_us":6129,"threads_started":1}
I20260925 09:20:32.785753 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.762848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41854)
0925 09:20:32.769297 (+ 6449us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.769318 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.769536 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.770118 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.770123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.770891 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.771219 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.771232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.772739 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.772744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.774461 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.774479 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.775818 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.775831 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.776123 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.776811 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.776838 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.778391 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.782483 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.782491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.782496 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.783049 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.783614 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.783618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.783623 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.783828 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.784518 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.784521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.784658 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.784824 (+ 166us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.785200 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6297,"thread_start_us":405,"threads_started":1}
I20260925 09:20:32.785583 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.763245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.3.82.61:41277)
0925 09:20:32.763450 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.763457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.769413 (+ 5956us) server_negotiation.cc:407] Connection header received
0925 09:20:32.769680 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.769687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.769773 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.769872 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.771401 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.772529 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.774636 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.775380 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.775492 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.778618 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.778628 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.778634 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.778682 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.782271 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.783214 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.783222 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.783226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.783397 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.784023 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.784028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.784030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.784338 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.784455 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.784931 (+ 476us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.785179 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:20:32.787461 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9627
I20260925 09:20:32.800089 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.764214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.3.82.60:45463)
0925 09:20:32.764414 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.764423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.776371 (+ 11948us) server_negotiation.cc:407] Connection header received
0925 09:20:32.776814 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.776820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.776969 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.777069 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.778780 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.780052 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.782691 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.783490 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.783590 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.790554 (+ 6964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.790564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.790572 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.790623 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.795919 (+ 5296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.797079 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.797087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.797093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.797308 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.797787 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.797797 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.797803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.798338 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.798518 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.799187 (+ 669us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.799516 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 09:20:32.805800 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.763764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.0.0.1:48420)
0925 09:20:32.776265 (+ 12501us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.776286 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.776643 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.777293 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.777299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.778195 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.778597 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.778610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.780266 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.780271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.782528 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.782544 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.786379 (+ 3835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.786391 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.786588 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.787496 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.787551 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.789400 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.796191 (+ 6791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.796199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.796206 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.796914 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.797473 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.797480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.797486 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.797660 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.798603 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.798613 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.798839 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.804541 (+ 5702us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.805020 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12340,"mutex_wait_us":1,"thread_start_us":12626,"threads_started":1}
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328029, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.62@KRBTEST.COM for oryx/127.3.82.60@KRBTEST.COM
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.1@KRBTEST.COM
I20260925 09:20:32.826444 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.789805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.0.0.1:48428)
0925 09:20:32.790021 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.790044 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.790178 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.801299 (+ 11121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.801308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.802046 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.802350 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.802368 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.806978 (+ 4610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.806990 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.809014 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.809041 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.810649 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.810659 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.810957 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.811847 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.811881 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.816048 (+ 4167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.822379 (+ 6331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.822389 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.822396 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.823147 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.823744 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.823751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.823758 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.823969 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.825266 (+ 1297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.825271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.825464 (+ 193us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.825626 (+ 162us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.825976 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 09:20:32.826431 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.789869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.3.82.60:45463)
0925 09:20:32.800724 (+ 10855us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.800732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.800755 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:20:32.800887 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.800910 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.801032 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.802553 (+ 1521us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.802801 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.804368 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.809337 (+ 4969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.810402 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.816326 (+ 5924us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.816687 (+ 361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.816696 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.816704 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.816746 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.822124 (+ 5378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.823378 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.823386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.823393 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.823578 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.824142 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.824149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.824155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.824631 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.825282 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.825735 (+ 453us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.826014 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10706,"thread_start_us":35056,"threads_started":1}
I20260925 09:20:32.828150 9566 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.82.60:45463: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c429f23af11e4acba958cf1dcde71bef"
member_type: NON_VOTER
last_known_addr {
host: "127.3.82.60"
port: 45463
}
attrs {
promote: true
}
}
dest_uuid: "02ef09e5832b484f871ed069a66f2770"
cas_config_opid_index: 6
I20260925 09:20:32.828835 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.788683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53748 (local address 127.3.82.1:36029)
0925 09:20:32.805211 (+ 16528us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.805222 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.805263 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:20:32.805340 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.805350 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.805680 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.809397 (+ 3717us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.809677 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.811382 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.813486 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.814472 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.815271 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.820559 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.820584 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.820612 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.820675 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.825105 (+ 4430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.826196 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.826207 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.826211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.826388 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.826876 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.826881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.826885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.827342 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.827660 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.828126 (+ 466us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.828406 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16268,"thread_start_us":17188,"threads_started":1}
I20260925 09:20:32.829046 9566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } }
I20260925 09:20:32.831629 9750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:32.831921 10184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Refusing update from remote peer 02ef09e5832b484f871ed069a66f2770: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 09:20:32.832085 10267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:32.832614 10128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 09:20:32.835646 10128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER c429f23af11e4acba958cf1dcde71bef (127.3.82.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:32.837502 10267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Refusing update from remote peer 02ef09e5832b484f871ed069a66f2770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 09:20:32.839434 10128 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.82.60:45463
I20260925 09:20:32.840145 10130 master.cc:537] Master@127.3.82.60:45463 shutting down...
I20260925 09:20:32.840374 10127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:32.843742 10184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER c429f23af11e4acba958cf1dcde71bef (127.3.82.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
W20260925 09:20:32.844635 9550 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260925 09:20:32.848562 10129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:32.848899 10129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:32.849223 10129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:32.849454 10129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:32.850135 10212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:32.850467 10212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 09:20:32.852981 9550 consensus_peers.cc:487] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 -> Peer c429f23af11e4acba958cf1dcde71bef (127.3.82.60:45463): Couldn't send request to peer c429f23af11e4acba958cf1dcde71bef. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:20:32.876274 10130 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:32.876760 10130 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:32.876974 10130 tablet_replica.cc:330] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: stopping tablet replica
I20260925 09:20:32.884394 10130 master.cc:552] Master@127.3.82.60:45463 shutdown complete.
I20260925 09:20:32.884500 10130 master_runner.cc:305] Clearing existing system tablet
I20260925 09:20:32.884900 10130 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:32.885017 10130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:32.891659 10130 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:20:32.893463 10323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:32.893899 10130 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:32.894348 10130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260925 09:20:32.894510 10130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
uuid: "c429f23af11e4acba958cf1dcde71bef"
format_stamp: "Formatted at 2026-09-25 09:20:32 on dist-test-slave-4wm5"
server_key: "bdcbc98571a870a68b3d551a1103e9d9"
server_key_iv: "6a7432bcfd748b4a82cf9fe2b0458727"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.900153 10130 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:32.907722 10130 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 09:20:32.908069 10130 log.cc:1196] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 09:20:32.908633 10130 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Deleting consensus metadata
I20260925 09:20:32.908972 10130 master_runner.cc:315] Copying system tablet from 127.3.82.62:36591
I20260925 09:20:32.911772 10130 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: tablet copy: Beginning tablet copy session from remote peer at address 127.3.82.62:36591
I20260925 09:20:32.928434 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.912755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.3.82.62:36591)
0925 09:20:32.912997 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:32.913004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:32.913488 (+ 484us) server_negotiation.cc:407] Connection header received
0925 09:20:32.913767 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:32.913773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:32.913884 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:32.913956 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:32.915206 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.916320 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.918273 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.918941 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.919070 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.921619 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:32.921626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:32.921631 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:32.921675 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:32.925243 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.926121 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.926127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.926131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.926288 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:32.926841 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:32.926846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:32.926850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:32.927216 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:32.927364 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:32.927847 (+ 483us) server_negotiation.cc:299] Negotiation successful
0925 09:20:32.928086 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260925 09:20:32.928673 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:32.912678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:58678)
0925 09:20:32.913369 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:32.913394 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:32.913608 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:32.914155 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:32.914161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:32.914674 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:32.915022 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.915036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.916475 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.916481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.918039 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:32.918056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:32.919109 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:32.919115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:32.919317 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:32.920142 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:32.920166 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:32.921368 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:32.925406 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.925412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.925416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.925980 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.926410 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:32.926415 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:32.926419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:32.926710 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:32.927350 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:32.927355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:32.927554 (+ 199us) client_negotiation.cc:769] Sending connection context
0925 09:20:32.927719 (+ 165us) client_negotiation.cc:240] Negotiation successful
0925 09:20:32.928091 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":422,"threads_started":1}
I20260925 09:20:32.929620 9586 tablet_copy_service.cc:140] P 02ef09e5832b484f871ed069a66f2770: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c429f23af11e4acba958cf1dcde71bef ({username='oryx', principal='oryx/127.3.82.60@KRBTEST.COM'} at 127.0.0.1:58678)
I20260925 09:20:32.929801 9586 tablet_copy_service.cc:161] P 02ef09e5832b484f871ed069a66f2770: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c429f23af11e4acba958cf1dcde71bef at {username='oryx', principal='oryx/127.3.82.60@KRBTEST.COM'} at 127.0.0.1:58678: session id = c429f23af11e4acba958cf1dcde71bef-00000000000000000000000000000000
I20260925 09:20:32.932188 9586 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 09:20:32.934223 10130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 09:20:32.941385 10130 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: tablet copy: Starting download of 0 data blocks...
I20260925 09:20:32.941726 10130 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: tablet copy: Starting download of 1 WAL segments...
I20260925 09:20:32.944422 10130 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 09:20:32.949074 9586 tablet_copy_service.cc:342] P 02ef09e5832b484f871ed069a66f2770: Request end of tablet copy session c429f23af11e4acba958cf1dcde71bef-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.82.60@KRBTEST.COM'} at 127.0.0.1:58678
I20260925 09:20:32.949244 9586 tablet_copy_service.cc:433] P 02ef09e5832b484f871ed069a66f2770: ending tablet copy session c429f23af11e4acba958cf1dcde71bef-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c429f23af11e4acba958cf1dcde71bef
I20260925 09:20:32.962572 10130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:32.965600 10330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:32.966233 10331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:32.974357 10130 server_base.cc:1034] running on GCE node
W20260925 09:20:32.974647 10334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:32.975157 10130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:32.975916 10130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:32.977144 10130 hybrid_clock.cc:648] HybridClock initialized: now 1790328032977122 us; error 76 us; skew 500 ppm
I20260925 09:20:32.977401 10130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:32 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328032, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:32.979688 10130 init.cc:377] Logged in from keytab as oryx/127.3.82.60@KRBTEST.COM (short username oryx)
I20260925 09:20:32.986402 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9627
I20260925 09:20:32.987593 10130 webserver.cc:460] Webserver started at http://127.3.82.60:40227/ using document root <none> and password file <none>
I20260925 09:20:32.987987 10130 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:32.988101 10130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:32.991771 10130 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260925 09:20:32.993669 10340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:32.994225 10130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260925 09:20:32.994383 10130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
uuid: "c429f23af11e4acba958cf1dcde71bef"
format_stamp: "Formatted at 2026-09-25 09:20:32 on dist-test-slave-4wm5"
server_key: "bdcbc98571a870a68b3d551a1103e9d9"
server_key_iv: "6a7432bcfd748b4a82cf9fe2b0458727"
server_key_version: "encryptionkey@0"
I20260925 09:20:32.994587 10130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:33.007647 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:36029
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:36591
--webserver_port=37255
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:20:33.041363 10130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:33.045626 10130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:33.076232 10130 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.60:45463
I20260925 09:20:33.076295 10392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.60:45463 every 8 connection(s)
I20260925 09:20:33.077212 10130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
I20260925 09:20:33.082301 10393 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:20:33.086791 10393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Bootstrap starting.
I20260925 09:20:33.107188 10393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:20:33.107790 10393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Bootstrap complete.
I20260925 09:20:33.108983 10393 raft_consensus.cc:348] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } }
I20260925 09:20:33.109313 10393 raft_consensus.cc:729] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c429f23af11e4acba958cf1dcde71bef, State: Initialized, Role: LEARNER
I20260925 09:20:33.109622 10393 consensus_queue.cc:260] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } }
I20260925 09:20:33.110620 10396 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:33.110903 10393 sys_catalog.cc:564] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:33.111092 10396 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: This master's current role is: LEARNER
I20260925 09:20:33.121500 10407 catalog_manager.cc:1259] Loaded cluster ID: 15326be2d0954770a1c5730c683877ee
I20260925 09:20:33.121624 10407 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: loading cluster ID for follower catalog manager: success
I20260925 09:20:33.125317 10407 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: acquiring CA information for follower catalog manager: success
I20260925 09:20:33.127888 10407 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:20:33.377604 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.369160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.0.0.1:48430)
0925 09:20:33.369615 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:33.369632 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:33.369778 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:33.370686 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:33.370692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:33.370725 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:33.370935 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.370949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.372760 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.372768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.374918 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.374938 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.376675 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.376681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.376911 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.376948 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:20:33.377086 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 09:20:33.377253 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":261,"threads_started":1}
I20260925 09:20:33.378002 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.369649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.3.82.60:45463)
0925 09:20:33.370300 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:33.370306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:33.370330 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:20:33.370411 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:33.370416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:33.370499 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:33.370628 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:33.371105 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.372579 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.375116 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.376458 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.376588 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.376830 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:33.377513 (+ 683us) server_negotiation.cc:299] Negotiation successful
0925 09:20:33.377661 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":374,"threads_started":1}
I20260925 09:20:33.379258 10368 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260925 09:20:33.379948 10396 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ef09e5832b484f871ed069a66f2770. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:33.380275 10396 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: This master's current role is: LEARNER
W20260925 09:20:33.407882 10345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:33.408255 10345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:33.408442 10345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:33.416437 10345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:33.416630 10345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:33.416810 10345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:33.427277 10345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:36029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=37255
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:33.428843 10345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:33.430491 10345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:33.445560 10417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:33.445587 10415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:33.445634 10414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:33.451990 10345 server_base.cc:1034] running on GCE node
I20260925 09:20:33.452803 10345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:33.454027 10345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:33.455279 10345 hybrid_clock.cc:648] HybridClock initialized: now 1790328033455197 us; error 69 us; skew 500 ppm
I20260925 09:20:33.455675 10345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:33 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328033, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:33.459720 10345 init.cc:377] Logged in from keytab as oryx/127.3.82.1@KRBTEST.COM (short username oryx)
I20260925 09:20:33.461539 10345 webserver.cc:460] Webserver started at http://127.3.82.1:37255/ using document root <none> and password file <none>
I20260925 09:20:33.462445 10345 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:33.462572 10345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:33.468801 10345 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:20:33.472282 10424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:33.474282 10345 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:33.474587 10345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "a170fc17915b1b9b6e1080545a48a50c"
server_key_iv: "d127b3a98a0fe409a2eae8ecc1971ea5"
server_key_version: "encryptionkey@0"
I20260925 09:20:33.475740 10345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:33.494156 10345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:33.499163 10345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:33.499420 10345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:33.500416 10345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:33.501976 10345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:33.502065 10345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:33.502141 10345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:33.502198 10345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:33.538194 10345 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:36029
I20260925 09:20:33.538236 10536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:36029 every 8 connection(s)
I20260925 09:20:33.539785 10345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:33.542156 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 10345
I20260925 09:20:33.542512 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9761
Sep 25 09:20:33 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328033, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.60@KRBTEST.COM
Sep 25 09:20:33 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.2@KRBTEST.COM
Sep 25 09:20:33 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328033, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:33.592201 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.550110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:40625 (local address 127.3.82.60:45463)
0925 09:20:33.550284 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:33.550291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:33.565301 (+ 15010us) server_negotiation.cc:407] Connection header received
0925 09:20:33.568629 (+ 3328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:33.568635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:33.568721 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:33.568785 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:33.571683 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.573046 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.576682 (+ 3636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.577612 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.577741 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.583368 (+ 5627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:33.583374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:33.583377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:33.583415 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:33.587099 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.588253 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.588261 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.588268 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.588426 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.589029 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.589038 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.589044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.589408 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:33.589542 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:33.591503 (+ 1961us) server_negotiation.cc:299] Negotiation successful
0925 09:20:33.591754 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:20:33.593338 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.543499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.3.82.2:44877)
0925 09:20:33.558948 (+ 15449us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:33.558962 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:33.566216 (+ 7254us) server_negotiation.cc:407] Connection header received
0925 09:20:33.566368 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:33.566382 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:33.566802 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:33.567022 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:33.570755 (+ 3733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.572366 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.574731 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.575920 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.576076 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.584050 (+ 7974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:33.584072 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:33.584092 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:33.584151 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:33.588981 (+ 4830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.590002 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.590013 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.590017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.590185 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.590791 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.590796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.590800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.591284 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:33.591608 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:33.592581 (+ 973us) server_negotiation.cc:299] Negotiation successful
0925 09:20:33.592870 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15202,"thread_start_us":18333,"threads_started":1}
Sep 25 09:20:33 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328033, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
I20260925 09:20:33.595630 9884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:33.596421 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.549511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.3.82.1:40625)
0925 09:20:33.565125 (+ 15614us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:33.565148 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:33.568448 (+ 3300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:33.569308 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:33.569321 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:33.570421 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:33.571399 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.571432 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.573318 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.573327 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.576433 (+ 3106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.576457 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.578233 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.578243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.578649 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.579652 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:33.579685 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:33.583088 (+ 3403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:33.587289 (+ 4201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:33.587300 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:33.587327 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:33.588094 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:33.588588 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:33.588593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:33.588599 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:33.588871 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:33.590452 (+ 1581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:33.590462 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:33.591003 (+ 541us) client_negotiation.cc:769] Sending connection context
0925 09:20:33.592438 (+ 1435us) client_negotiation.cc:240] Negotiation successful
0925 09:20:33.593206 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15465,"thread_start_us":15789,"threads_started":1}
I20260925 09:20:33.598968 10538 heartbeater.cc:347] Connected to a master server at 127.3.82.60:45463
I20260925 09:20:33.599345 10538 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:33.600363 10538 heartbeater.cc:511] Master 127.3.82.60:45463 requested a full tablet report, sending...
I20260925 09:20:33.603245 10358 ts_manager.cc:194] Registered new tserver with Master: 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029)
I20260925 09:20:33.603322 10546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.573059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:46017 (local address 127.3.82.61:41277)
0925 09:20:33.579420 (+ 6361us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:33.579427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:33.585709 (+ 6282us) server_negotiation.cc:407] Connection header received
0925 09:20:33.586181 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:33.586187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:33.586274 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:33.586354 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:33.587896 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.589066 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.590699 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.591339 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.591455 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.596226 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:33.596232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:33.596235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:33.596273 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:33.599876 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.600883 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.600889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.600893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.601067 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.601532 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.601538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.601541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.601963 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:33.602134 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:33.602703 (+ 569us) server_negotiation.cc:299] Negotiation successful
0925 09:20:33.602950 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6191,"thread_start_us":6060,"threads_started":1}
I20260925 09:20:33.604573 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.574620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.3.82.1:46017)
0925 09:20:33.585598 (+ 10978us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:33.585619 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:33.586010 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:33.586621 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:33.586625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:33.587324 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:33.587688 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.587702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.589251 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.589254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.590547 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.590558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.591728 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.591735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.591916 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.592774 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:33.592796 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:33.595934 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:33.600078 (+ 4144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:33.600083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:33.600087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:33.600714 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:33.601226 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:33.601231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:33.601234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:33.601402 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:33.602199 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:33.602204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:33.602389 (+ 185us) client_negotiation.cc:769] Sending connection context
0925 09:20:33.603756 (+ 1367us) client_negotiation.cc:240] Negotiation successful
0925 09:20:33.604146 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10815,"thread_start_us":11104,"threads_started":1}
I20260925 09:20:33.605609 10539 heartbeater.cc:347] Connected to a master server at 127.3.82.61:41277
I20260925 09:20:33.605779 10539 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:33.606302 10539 heartbeater.cc:511] Master 127.3.82.61:41277 requested a full tablet report, sending...
I20260925 09:20:33.606550 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.544586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:47163)
0925 09:20:33.559750 (+ 15164us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:33.559828 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:33.567928 (+ 8100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:33.579531 (+ 11603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:33.579539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:33.580103 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:33.580446 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.580461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.582138 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.582147 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.585607 (+ 3460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:33.585623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.586909 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.586921 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:33.587137 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.587849 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:33.587875 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:33.591406 (+ 3531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:33.601666 (+ 10260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:33.601675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:33.601681 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:33.602356 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:33.602969 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:33.602977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:33.603001 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:33.603197 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:33.604621 (+ 1424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:33.604937 (+ 316us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:33.605146 (+ 209us) client_negotiation.cc:769] Sending connection context
0925 09:20:33.605320 (+ 174us) client_negotiation.cc:240] Negotiation successful
0925 09:20:33.605754 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14719,"thread_start_us":14463,"threads_started":1}
I20260925 09:20:33.607848 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:33.545368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:47163 (local address 127.3.82.62:36591)
0925 09:20:33.564532 (+ 19164us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:33.564543 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:33.564581 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:20:33.571676 (+ 7095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:33.571683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:33.571767 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:33.571874 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:33.580732 (+ 8858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.581967 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.585805 (+ 3838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:33.586525 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:33.586645 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:33.591661 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:33.591671 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:33.591678 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:33.591725 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:33.601539 (+ 9814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.602552 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.602561 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.602568 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.602810 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:33.603347 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:33.603772 (+ 425us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:33.603792 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:33.604303 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:33.604518 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:33.606999 (+ 2481us) server_negotiation.cc:299] Negotiation successful
0925 09:20:33.607345 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18951,"thread_start_us":19212,"threads_started":1}
I20260925 09:20:33.608340 10174 ts_manager.cc:194] Registered new tserver with Master: 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029)
I20260925 09:20:33.608716 10537 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36591
I20260925 09:20:33.608855 10537 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:33.609350 10537 heartbeater.cc:511] Master 127.3.82.62:36591 requested a full tablet report, sending...
I20260925 09:20:33.610670 9566 ts_manager.cc:194] Re-registered known tserver with Master: 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029)
I20260925 09:20:33.611852 9566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:47163
I20260925 09:20:33.698969 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9761
I20260925 09:20:33.716930 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:44877
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:36591
--webserver_port=43679
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:20:33.935631 10128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: NON_VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: true } } }
I20260925 09:20:33.936045 10128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:33.937489 10551 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: attempting to promote NON_VOTER c429f23af11e4acba958cf1dcde71bef to VOTER
I20260925 09:20:33.938176 10551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } }
I20260925 09:20:33.940508 10184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Refusing update from remote peer 02ef09e5832b484f871ed069a66f2770: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 09:20:33.940501 10368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 LEARNER]: Refusing update from remote peer 02ef09e5832b484f871ed069a66f2770: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 09:20:33.941179 10128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 09:20:33.941668 10550 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 09:20:33.946332 10551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c429f23af11e4acba958cf1dcde71bef (127.3.82.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.947036 10184 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c429f23af11e4acba958cf1dcde71bef (127.3.82.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.947247 10368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c429f23af11e4acba958cf1dcde71bef (127.3.82.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.950911 10551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.951359 10551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:33.951846 10550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.952143 10550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:33.951877 10551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.952385 10551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:33.953056 10556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.953694 10556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:20:33.954218 10557 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02ef09e5832b484f871ed069a66f2770" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:33.954568 10557 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: This master's current role is: FOLLOWER
W20260925 09:20:34.076745 10549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:34.077097 10549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:34.077272 10549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:34.085616 10549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:34.085777 10549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:34.086010 10549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:34.097357 10549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:44877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=43679
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:34.099085 10549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:34.100595 10549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:34.116793 10562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:34.117024 10565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:34.117960 10549 server_base.cc:1034] running on GCE node
W20260925 09:20:34.118043 10563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:34.118845 10549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:34.120111 10549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:34.121387 10549 hybrid_clock.cc:648] HybridClock initialized: now 1790328034121321 us; error 53 us; skew 500 ppm
I20260925 09:20:34.121902 10549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:34.125633 10549 init.cc:377] Logged in from keytab as oryx/127.3.82.2@KRBTEST.COM (short username oryx)
I20260925 09:20:34.127677 10549 webserver.cc:460] Webserver started at http://127.3.82.2:43679/ using document root <none> and password file <none>
I20260925 09:20:34.128485 10549 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:34.128672 10549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:34.135504 10549 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:20:34.138924 10572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:34.140739 10549 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:20:34.140971 10549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa"
format_stamp: "Formatted at 2026-09-25 09:20:30 on dist-test-slave-4wm5"
server_key: "2b460fc481fc69a0bd44523ce26cb10c"
server_key_iv: "d950cc98ebe7314b6e69763efc01b2df"
server_key_version: "encryptionkey@0"
I20260925 09:20:34.142302 10549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:34.164819 10549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:34.170085 10549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:34.170503 10549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:34.171749 10549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:34.173308 10549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:34.173406 10549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:34.173501 10549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:34.173561 10549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:34.212090 10549 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:44877
I20260925 09:20:34.212276 10684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:44877 every 8 connection(s)
I20260925 09:20:34.214035 10549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:34.221477 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 10549
I20260925 09:20:34.221863 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9895
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.3@KRBTEST.COM
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.60@KRBTEST.COM
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
I20260925 09:20:34.256374 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.223758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.3.82.3:39133)
0925 09:20:34.228187 (+ 4429us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.228201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.228237 (+ 36us) server_negotiation.cc:407] Connection header received
0925 09:20:34.228327 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.228340 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.228765 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.228979 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.232214 (+ 3235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.234212 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.236758 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.237930 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.238119 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.245240 (+ 7121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.245263 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.245285 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.245336 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.251165 (+ 5829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.252405 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.252418 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.252424 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.252625 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.253191 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.253199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.253203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.253721 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.254084 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.255564 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.255890 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4210,"thread_start_us":4032,"threads_started":1}
I20260925 09:20:34.258132 10018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:34.259563 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.217466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41035 (local address 127.3.82.62:36591)
0925 09:20:34.236589 (+ 19123us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.236596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.236617 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:20:34.236694 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.236699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.236780 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.237248 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.239176 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.239987 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.242989 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.243556 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.243706 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.249271 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.249277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.249280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.249330 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.253935 (+ 4605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.254882 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.254892 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.254898 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.255080 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.257684 (+ 2604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.257690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.257694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.258116 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.258285 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.258862 (+ 577us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.259154 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18982,"thread_start_us":17821,"threads_started":1}
I20260925 09:20:34.259761 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.217426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:51053 (local address 127.3.82.61:41277)
0925 09:20:34.235847 (+ 18421us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.235852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.235871 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:20:34.235930 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.235933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.236006 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.236163 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.241551 (+ 5388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.242802 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.246034 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.246810 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.246944 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.251416 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.251424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.251429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.251473 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.255482 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.256837 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.256846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.256852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.256993 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.257380 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.257385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.257389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.257788 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.257989 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.258900 (+ 911us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.259256 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18287,"thread_start_us":18857,"threads_started":1}
I20260925 09:20:34.261070 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.230444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:33013 (local address 127.3.82.60:45463)
0925 09:20:34.238556 (+ 8112us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.238563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.238582 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:20:34.238642 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.238647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.238737 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.238858 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.240167 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.241445 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.244011 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.244845 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.244978 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.251417 (+ 6439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.251424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.251429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.251473 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.255439 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.256504 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.256511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.256514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.256659 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.257126 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.257132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.257135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.257539 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.257719 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.260343 (+ 2624us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.260640 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7975,"thread_start_us":7871,"threads_started":1}
I20260925 09:20:34.261809 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.218324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:41035)
0925 09:20:34.219631 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.219704 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.223473 (+ 3769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.237003 (+ 13530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.237008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.237898 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:34.238934 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.238996 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.240171 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.240177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.242797 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.242813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.243819 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.243827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.244235 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.245149 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:34.245183 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:34.248954 (+ 3771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:34.254180 (+ 5226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.254188 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.254203 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.254686 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.256105 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.256112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.256117 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.256623 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.258312 (+ 1689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:34.258317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:34.258495 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.258737 (+ 242us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.259372 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":515,"threads_started":1}
I20260925 09:20:34.262578 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.232876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.3.82.2:33013)
0925 09:20:34.237942 (+ 5066us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.237965 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.238108 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.238932 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.238939 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.239620 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:34.239934 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.239951 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.241665 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.241672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.243387 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.243405 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.245241 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.245248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.245484 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.246647 (+ 1163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:34.246675 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:34.251135 (+ 4460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:34.255630 (+ 4495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.255636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.255640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.256331 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.256793 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.256799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.256802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.256961 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.257806 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:34.257815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:34.260022 (+ 2207us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.260204 (+ 182us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.262038 (+ 1834us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4900,"mutex_wait_us":1378,"thread_start_us":4713,"threads_started":1}
I20260925 09:20:34.263842 10685 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36591
I20260925 09:20:34.264048 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.218324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.3.82.2:51053)
0925 09:20:34.230166 (+ 11842us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.230198 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.230378 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.236587 (+ 6209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.236603 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.240951 (+ 4348us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:34.241311 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.241324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.244004 (+ 2680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.244010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.245869 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.245884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.247350 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.247357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.247544 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.248083 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:34.248104 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:34.251192 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:34.255639 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.255643 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.255646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.256688 (+ 1042us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.257130 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.257135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.257138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.257253 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.258078 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:34.258082 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:34.258374 (+ 292us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.259964 (+ 1590us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.260435 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11571,"mutex_wait_us":479,"spinlock_wait_cycles":66816,"thread_start_us":10816,"threads_started":1}
I20260925 09:20:34.264287 10685 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:34.265232 10687 heartbeater.cc:347] Connected to a master server at 127.3.82.61:41277
I20260925 09:20:34.265372 10685 heartbeater.cc:511] Master 127.3.82.62:36591 requested a full tablet report, sending...
I20260925 09:20:34.265394 10687 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:34.265990 10687 heartbeater.cc:511] Master 127.3.82.61:41277 requested a full tablet report, sending...
I20260925 09:20:34.266227 10686 heartbeater.cc:347] Connected to a master server at 127.3.82.60:45463
I20260925 09:20:34.266345 10686 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:34.266846 10686 heartbeater.cc:511] Master 127.3.82.60:45463 requested a full tablet report, sending...
I20260925 09:20:34.267663 10174 ts_manager.cc:194] Registered new tserver with Master: 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877)
I20260925 09:20:34.268069 10358 ts_manager.cc:194] Registered new tserver with Master: 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877)
I20260925 09:20:34.269028 9566 ts_manager.cc:194] Re-registered known tserver with Master: 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877)
I20260925 09:20:34.270103 9566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:41035
I20260925 09:20:34.356655 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9895
I20260925 09:20:34.372990 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:39133
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:36591
--webserver_port=37841
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:20:34.614962 10537 heartbeater.cc:503] Master 127.3.82.62:36591 was elected leader, sending a full tablet report...
W20260925 09:20:34.726312 10698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:34.726636 10698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:34.726802 10698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:34.734930 10698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:34.735061 10698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:34.735251 10698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:34.745752 10698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:39133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=37841
--tserver_master_addrs=127.3.82.62:36591,127.3.82.61:41277,127.3.82.60:45463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:34.747341 10698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:34.748751 10698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:34.764128 10703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:34.764348 10704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:34.764907 10706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:34.764834 10698 server_base.cc:1034] running on GCE node
I20260925 09:20:34.765700 10698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:34.766803 10698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:34.768046 10698 hybrid_clock.cc:648] HybridClock initialized: now 1790328034767968 us; error 67 us; skew 500 ppm
I20260925 09:20:34.768378 10698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:34.776312 10698 init.cc:377] Logged in from keytab as oryx/127.3.82.3@KRBTEST.COM (short username oryx)
I20260925 09:20:34.778126 10698 webserver.cc:460] Webserver started at http://127.3.82.3:37841/ using document root <none> and password file <none>
I20260925 09:20:34.778900 10698 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:34.779062 10698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:34.785730 10698 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 09:20:34.789099 10713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:34.791285 10698 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:34.791467 10698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "86845cbeecfb4beca94170ae87538a59"
format_stamp: "Formatted at 2026-09-25 09:20:31 on dist-test-slave-4wm5"
server_key: "8074407ad92a52fd2743d25afcbbadab"
server_key_iv: "ad088bb5e7f429427e43508c7cf80eb5"
server_key_version: "encryptionkey@0"
I20260925 09:20:34.792707 10698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:34.813069 10698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:34.818498 10698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:34.818825 10698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:34.819948 10698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:34.821462 10698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:34.821556 10698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:34.821719 10698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:34.821805 10698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:34.857568 10698 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:39133
I20260925 09:20:34.857623 10825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:39133 every 8 connection(s)
I20260925 09:20:34.859141 10698 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:34.866714 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 10698
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.61@KRBTEST.COM
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
Sep 25 09:20:34 dist-test-slave-4wm5 krb5kdc[9521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328034, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.60@KRBTEST.COM
I20260925 09:20:34.895653 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.862394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:49247 (local address 127.3.82.61:41277)
0925 09:20:34.862975 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.862982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.864498 (+ 1516us) server_negotiation.cc:407] Connection header received
0925 09:20:34.867323 (+ 2825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.867328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.867414 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.867510 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.870897 (+ 3387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.872059 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.875812 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.877002 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.877182 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.884011 (+ 6829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.884017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.884021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.884071 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.888918 (+ 4847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.890122 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.890129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.890133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.890318 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.890929 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.890937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.890941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.891401 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.891579 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.894906 (+ 3327us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.895193 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":364,"threads_started":1}
I20260925 09:20:34.897058 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.863092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.3.82.3:49247)
0925 09:20:34.864286 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.864356 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.866822 (+ 2466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.867941 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.867955 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.869301 (+ 1346us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:34.870612 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.870642 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.872326 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.872335 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.875512 (+ 3177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.875541 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.877675 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.877686 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.878178 (+ 492us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.879615 (+ 1437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:34.879675 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:34.883473 (+ 3798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:34.889124 (+ 5651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.889135 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.889156 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.889922 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.890475 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.890481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.890486 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.890775 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.891676 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:34.891688 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:34.892241 (+ 553us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.892707 (+ 466us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.893417 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":283,"spinlock_wait_cycles":73600,"thread_start_us":477,"threads_started":1}
I20260925 09:20:34.899151 10828 heartbeater.cc:347] Connected to a master server at 127.3.82.61:41277
I20260925 09:20:34.899660 10828 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:34.900818 10828 heartbeater.cc:511] Master 127.3.82.61:41277 requested a full tablet report, sending...
I20260925 09:20:34.903074 10174 ts_manager.cc:194] Registered new tserver with Master: 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133)
I20260925 09:20:34.904767 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.865138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50447 (local address 127.3.82.60:45463)
0925 09:20:34.865965 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.865970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.865989 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:20:34.867289 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.867295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.867390 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.867502 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.870870 (+ 3368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.872079 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.884874 (+ 12795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.885738 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.885906 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.892028 (+ 6122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.892035 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.892040 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.892084 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.896419 (+ 4335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.897729 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.897735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.897739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.897976 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.898501 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.898507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.898511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.898812 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.898993 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.904000 (+ 5007us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.904301 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":468,"threads_started":1}
I20260925 09:20:34.907829 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.863092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:52989)
0925 09:20:34.864286 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.864356 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.866822 (+ 2466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.867941 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.867955 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.869656 (+ 1701us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:34.870610 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.870642 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.878472 (+ 7830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.878481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.880449 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.880473 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.884189 (+ 3716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.884195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.884816 (+ 621us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.885679 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:34.885707 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:34.897992 (+ 12285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:34.904098 (+ 6106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.904104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.904107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.904654 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.905177 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.905182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.905188 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.905386 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.906600 (+ 1214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:34.906608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:34.906773 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.906920 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.907267 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":387,"spinlock_wait_cycles":105600,"thread_start_us":517,"threads_started":1}
I20260925 09:20:34.908867 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.864987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.3.82.3:50447)
0925 09:20:34.865605 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.865625 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.867030 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.867943 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.867956 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.869301 (+ 1345us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:34.870610 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.870642 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.872322 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.872327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.884631 (+ 12304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.884648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.886304 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.886312 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.886563 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.887214 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:34.887239 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:34.891774 (+ 4535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:34.896669 (+ 4895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.896675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.896679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.897545 (+ 866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.898158 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:34.898163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:34.898167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:34.898330 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:34.898975 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:34.903351 (+ 4376us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:34.903568 (+ 217us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.907740 (+ 4172us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.908408 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":267,"spinlock_wait_cycles":42880,"thread_start_us":304,"threads_started":1}
I20260925 09:20:34.908984 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.863013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:52989 (local address 127.3.82.62:36591)
0925 09:20:34.863486 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.863494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.864497 (+ 1003us) server_negotiation.cc:407] Connection header received
0925 09:20:34.867207 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.867213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.867306 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.867394 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.870869 (+ 3475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.872089 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.882968 (+ 10879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.883752 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.883856 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.899464 (+ 15608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.899473 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.899481 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.899529 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.903906 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.904837 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.904845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.904855 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.905028 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.905555 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.905561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.905564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.905995 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.906519 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.908241 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.908543 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":355,"thread_start_us":253,"threads_started":1}
I20260925 09:20:34.909919 10826 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36591
I20260925 09:20:34.910291 10826 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:34.910840 10826 heartbeater.cc:511] Master 127.3.82.62:36591 requested a full tablet report, sending...
I20260925 09:20:34.912117 9566 ts_manager.cc:194] Re-registered known tserver with Master: 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133)
I20260925 09:20:34.913643 9566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:52989
I20260925 09:20:34.914951 10827 heartbeater.cc:347] Connected to a master server at 127.3.82.60:45463
I20260925 09:20:34.915128 10827 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:34.916133 10827 heartbeater.cc:511] Master 127.3.82.60:45463 requested a full tablet report, sending...
I20260925 09:20:34.917604 10358 ts_manager.cc:194] Registered new tserver with Master: 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133)
I20260925 09:20:34.920967 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.874297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.3.82.61:41277)
0925 09:20:34.889365 (+ 15068us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.889375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.891734 (+ 2359us) server_negotiation.cc:407] Connection header received
0925 09:20:34.891816 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.891824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.891929 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.892061 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.895939 (+ 3878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.897439 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.902650 (+ 5211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.903373 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.903477 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.911227 (+ 7750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.911236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.911240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.911287 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.916292 (+ 5005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.918675 (+ 2383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.918682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.918686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.918862 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.919310 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.919315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.919319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.919623 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.919923 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.920291 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.920540 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14874,"thread_start_us":14691,"threads_started":1}
I20260925 09:20:34.925420 10841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.872404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.3.82.62:36591)
0925 09:20:34.898738 (+ 26334us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.898744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.898761 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:34.898817 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.898822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.898907 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.899030 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.903727 (+ 4697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.905177 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.907722 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.908635 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.908793 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.917586 (+ 8793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.917595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.917601 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.917670 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.921653 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.923245 (+ 1592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.923249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.923251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.923396 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.923812 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.923824 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.923826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.924122 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.924412 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.924807 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.925037 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":26193,"thread_start_us":26803,"threads_started":1}
I20260925 09:20:34.929301 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.890309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.3.82.60:45463)
0925 09:20:34.905440 (+ 15131us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.905449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.910459 (+ 5010us) server_negotiation.cc:407] Connection header received
0925 09:20:34.910844 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.910852 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.910943 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.911015 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:34.913403 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.914593 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.916738 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.917492 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.917688 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.922557 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:34.922566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:34.922574 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:34.922629 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:34.926156 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.927019 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.927024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.927026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.927165 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:34.927603 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:34.927610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:34.927614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:34.927936 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:34.928234 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.928632 (+ 398us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.928895 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14983,"thread_start_us":19829,"threads_started":1}
I20260925 09:20:34.930192 9566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:34.937431 9566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:34.969007 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.957193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:44877 (local address 127.0.0.1:47474)
0925 09:20:34.958410 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.958430 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.958673 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.960306 (+ 1633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.960314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.960352 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:34.960842 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.960858 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.963336 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.963340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.965490 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.965509 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.967958 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.967966 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.968165 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.968206 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.968411 (+ 205us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.968589 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"thread_start_us":1762,"threads_started":1}
I20260925 09:20:34.969866 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.957459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.3.82.2:44877)
0925 09:20:34.958050 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.958062 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.958543 (+ 481us) server_negotiation.cc:407] Connection header received
0925 09:20:34.958917 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.958927 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.959746 (+ 819us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.959932 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:34.961089 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.963149 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.965673 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.967425 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.967571 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.967804 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.969344 (+ 1540us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.969549 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":280,"threads_started":1}
I20260925 09:20:34.970625 10856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.958242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:39133 (local address 127.0.0.1:51954)
0925 09:20:34.958921 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.958939 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.959035 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.961973 (+ 2938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.961981 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.962022 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:34.962507 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.962518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.965380 (+ 2862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.965387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.967347 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.967363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.969610 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.969616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.969811 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.969909 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.970032 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.970183 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":45,"thread_start_us":361,"threads_started":1}
I20260925 09:20:34.971768 10857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.958507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.3.82.3:39133)
0925 09:20:34.959324 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.959333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.959365 (+ 32us) server_negotiation.cc:407] Connection header received
0925 09:20:34.959472 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.959481 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.959759 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.960010 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:34.963462 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.965207 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.967871 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.969333 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.969514 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.969755 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.971371 (+ 1616us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.971527 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":908,"threads_started":1}
I20260925 09:20:34.973949 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.959551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.3.82.1:36029)
0925 09:20:34.963795 (+ 4244us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:34.963804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:34.964691 (+ 887us) server_negotiation.cc:407] Connection header received
0925 09:20:34.964862 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:34.964870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:34.965175 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:34.965393 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:34.966490 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.968051 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.970595 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.972048 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.972203 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.972418 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:34.973178 (+ 760us) server_negotiation.cc:299] Negotiation successful
0925 09:20:34.973357 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4053,"thread_start_us":3248,"threads_started":1}
I20260925 09:20:34.974364 10620 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec6485409d84f1ba1da62679616f7c9 (DEFAULT_TABLE table=test-table [id=08bf1270a83b40168322458a64ad8229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:34.974870 10858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:34.959316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:36029 (local address 127.0.0.1:53760)
0925 09:20:34.964536 (+ 5220us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:34.964562 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:34.964703 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:34.965630 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:34.965634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:34.965666 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:34.966244 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.966258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.968235 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.968241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.970419 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:34.970431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:34.972273 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:34.972279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:34.972942 (+ 663us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:34.972988 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 09:20:34.974369 (+ 1381us) client_negotiation.cc:240] Negotiation successful
0925 09:20:34.974529 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5076,"thread_start_us":4903,"threads_started":1}
I20260925 09:20:34.975659 10620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec6485409d84f1ba1da62679616f7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:34.975737 10761 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec6485409d84f1ba1da62679616f7c9 (DEFAULT_TABLE table=test-table [id=08bf1270a83b40168322458a64ad8229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:34.977114 10761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec6485409d84f1ba1da62679616f7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:34.979012 10472 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec6485409d84f1ba1da62679616f7c9 (DEFAULT_TABLE table=test-table [id=08bf1270a83b40168322458a64ad8229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:34.980973 10472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec6485409d84f1ba1da62679616f7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:34.991510 10860 tablet_bootstrap.cc:492] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Bootstrap starting.
I20260925 09:20:34.993901 10861 tablet_bootstrap.cc:492] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Bootstrap starting.
I20260925 09:20:34.996062 10860 tablet_bootstrap.cc:654] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:34.997653 10860 log.cc:824] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:34.998003 10861 tablet_bootstrap.cc:654] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:34.999578 10861 log.cc:824] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:35.000825 10862 tablet_bootstrap.cc:492] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Bootstrap starting.
I20260925 09:20:35.002218 10860 tablet_bootstrap.cc:492] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: No bootstrap required, opened a new log
I20260925 09:20:35.002586 10860 ts_tablet_manager.cc:1397] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260925 09:20:35.002751 10861 tablet_bootstrap.cc:492] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: No bootstrap required, opened a new log
I20260925 09:20:35.003047 10861 ts_tablet_manager.cc:1397] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260925 09:20:35.005017 10862 tablet_bootstrap.cc:654] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:35.006695 10862 log.cc:824] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:35.011379 10862 tablet_bootstrap.cc:492] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: No bootstrap required, opened a new log
I20260925 09:20:35.011751 10862 ts_tablet_manager.cc:1397] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260925 09:20:35.012851 10860 raft_consensus.cc:348] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.012885 10861 raft_consensus.cc:348] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.013300 10860 raft_consensus.cc:374] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:35.013407 10860 raft_consensus.cc:729] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86845cbeecfb4beca94170ae87538a59, State: Initialized, Role: FOLLOWER
I20260925 09:20:35.013410 10861 raft_consensus.cc:374] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:35.013525 10861 raft_consensus.cc:729] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac4bcf51fec4ac3ad9018d16ff2f6fa, State: Initialized, Role: FOLLOWER
I20260925 09:20:35.014303 10861 consensus_queue.cc:260] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.014374 10860 consensus_queue.cc:260] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.016073 10685 heartbeater.cc:503] Master 127.3.82.62:36591 was elected leader, sending a full tablet report...
I20260925 09:20:35.016822 10861 ts_tablet_manager.cc:1428] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260925 09:20:35.025440 10862 raft_consensus.cc:348] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.026021 10862 raft_consensus.cc:374] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:35.026187 10862 raft_consensus.cc:729] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1ab20af64f4c5e8b8d12e4cdfb8880, State: Initialized, Role: FOLLOWER
I20260925 09:20:35.026984 10862 consensus_queue.cc:260] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.027683 10826 heartbeater.cc:503] Master 127.3.82.62:36591 was elected leader, sending a full tablet report...
I20260925 09:20:35.028568 10860 ts_tablet_manager.cc:1428] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Time spent starting tablet: real 0.026s user 0.023s sys 0.001s
I20260925 09:20:35.032827 10862 ts_tablet_manager.cc:1428] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Time spent starting tablet: real 0.021s user 0.017s sys 0.000s
I20260925 09:20:35.048405 10867 raft_consensus.cc:482] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:35.048723 10867 raft_consensus.cc:504] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.050573 10867 leader_election.cc:288] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877), 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029)
I20260925 09:20:35.056409 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.051039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:53531 (local address 127.3.82.2:44877)
0925 09:20:35.051229 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.051233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.051281 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:20:35.051556 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.051561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.051646 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.051771 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:35.052168 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.052888 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.054614 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.055466 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.055617 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.055713 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.056004 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.056138 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:20:35.056414 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.050998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:44877 (local address 127.3.82.3:53531)
0925 09:20:35.051187 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:35.051210 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:35.051381 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:35.051814 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:35.051819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:35.051848 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:35.052015 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:35.052023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.053031 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.053039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:35.054455 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:35.054468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.055640 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.055646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:35.055834 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.055859 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:20:35.055968 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:20:35.056130 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260925 09:20:35.056706 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.051770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:45973 (local address 127.3.82.1:36029)
0925 09:20:35.051968 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.051972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.051987 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:20:35.052044 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.052047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.052100 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.052209 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:35.052568 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.053271 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.054962 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.055755 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.055901 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.056000 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.056307 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.056448 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:20:35.056730 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.051435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:36029 (local address 127.3.82.3:45973)
0925 09:20:35.051631 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:35.051649 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:35.051741 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:35.052222 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:35.052227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:35.052249 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:35.052414 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:35.052421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.053424 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.053429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:35.054792 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:35.054805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.055935 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.055941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:35.056142 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.056170 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 09:20:35.056278 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:20:35.056418 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 09:20:35.057633 10640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6485409d84f1ba1da62679616f7c9" candidate_uuid: "86845cbeecfb4beca94170ae87538a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" is_pre_election: true
I20260925 09:20:35.057735 10492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6485409d84f1ba1da62679616f7c9" candidate_uuid: "86845cbeecfb4beca94170ae87538a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" is_pre_election: true
I20260925 09:20:35.058255 10640 raft_consensus.cc:2454] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86845cbeecfb4beca94170ae87538a59 in term 0.
I20260925 09:20:35.058308 10492 raft_consensus.cc:2454] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86845cbeecfb4beca94170ae87538a59 in term 0.
I20260925 09:20:35.059063 10714 leader_election.cc:302] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac4bcf51fec4ac3ad9018d16ff2f6fa, 86845cbeecfb4beca94170ae87538a59; no voters:
I20260925 09:20:35.059568 10867 raft_consensus.cc:2781] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:35.059727 10867 raft_consensus.cc:482] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:35.060006 10867 raft_consensus.cc:3037] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:35.062445 10867 raft_consensus.cc:504] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.063076 10867 leader_election.cc:288] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [CANDIDATE]: Term 1 election: Requested vote from peers 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877), 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029)
I20260925 09:20:35.063666 10492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6485409d84f1ba1da62679616f7c9" candidate_uuid: "86845cbeecfb4beca94170ae87538a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880"
I20260925 09:20:35.063692 10640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec6485409d84f1ba1da62679616f7c9" candidate_uuid: "86845cbeecfb4beca94170ae87538a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa"
I20260925 09:20:35.063889 10492 raft_consensus.cc:3037] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:35.063990 10640 raft_consensus.cc:3037] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:35.066371 10492 raft_consensus.cc:2454] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86845cbeecfb4beca94170ae87538a59 in term 1.
I20260925 09:20:35.066898 10714 leader_election.cc:302] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1ab20af64f4c5e8b8d12e4cdfb8880, 86845cbeecfb4beca94170ae87538a59; no voters:
I20260925 09:20:35.066852 10640 raft_consensus.cc:2454] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86845cbeecfb4beca94170ae87538a59 in term 1.
I20260925 09:20:35.067308 10867 raft_consensus.cc:2781] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:35.067926 10867 raft_consensus.cc:686] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 1 LEADER]: Becoming Leader. State: Replica: 86845cbeecfb4beca94170ae87538a59, State: Running, Role: LEADER
I20260925 09:20:35.068605 10867 consensus_queue.cc:237] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } }
I20260925 09:20:35.074349 9565 catalog_manager.cc:5495] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86845cbeecfb4beca94170ae87538a59 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "86845cbeecfb4beca94170ae87538a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } health_report { overall_health: HEALTHY } } }
W20260925 09:20:35.080833 10545 tablet.cc:2369] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:35.097147 10857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.091096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.3.82.3:39133)
0925 09:20:35.091330 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.091337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.091361 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:20:35.091673 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.091677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.091762 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.091951 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:35.092362 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.093104 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.094671 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.095165 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.095306 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.095854 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:35.096495 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:35.096704 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.096767 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.096857 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:20:35.106158 10640 raft_consensus.cc:1261] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Refusing update from remote peer 86845cbeecfb4beca94170ae87538a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:35.106207 10492 raft_consensus.cc:1261] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Refusing update from remote peer 86845cbeecfb4beca94170ae87538a59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:35.107372 10869 consensus_queue.cc:1035] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:35.107908 10867 consensus_queue.cc:1035] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:20:35.111728 10829 tablet.cc:2369] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:35.128628 10874 mvcc.cc:205] Tried to move back new op lower bound from 7333183631783440384 to 7333183631645286400. Current Snapshot: MvccSnapshot[applied={T|T < 7333183631783440384 or (T in {7333183631783440384})}]
I20260925 09:20:35.149812 9565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:35.150192 9565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:35.158001 9565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02ef09e5832b484f871ed069a66f2770: Sending DeleteTablet for 3 replicas of tablet 5ec6485409d84f1ba1da62679616f7c9
I20260925 09:20:35.160071 10620 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ec6485409d84f1ba1da62679616f7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:35 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:47474
I20260925 09:20:35.160043 10761 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ec6485409d84f1ba1da62679616f7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:35 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:51954
I20260925 09:20:35.160918 10472 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ec6485409d84f1ba1da62679616f7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:35 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53760
I20260925 09:20:35.168624 10885 tablet_replica.cc:330] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: stopping tablet replica
I20260925 09:20:35.169655 10885 raft_consensus.cc:2229] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:35.172465 10885 raft_consensus.cc:2258] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:35.175678 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 9534
I20260925 09:20:35.177057 9596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:35.178973 10885 ts_tablet_manager.cc:1905] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:35.175123 10884 tablet_replica.cc:330] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: stopping tablet replica
I20260925 09:20:35.179901 10884 raft_consensus.cc:2229] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:35.180827 10884 raft_consensus.cc:2258] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:35.185631 10884 ts_tablet_manager.cc:1905] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:35.190922 10883 tablet_replica.cc:330] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: stopping tablet replica
I20260925 09:20:35.191712 10883 raft_consensus.cc:2229] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:35.194458 10883 raft_consensus.cc:2258] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:35.198097 10883 ts_tablet_manager.cc:1905] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:35.203778 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.162528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.3.82.61:41277)
0925 09:20:35.162735 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.162742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.173557 (+ 10815us) server_negotiation.cc:407] Connection header received
0925 09:20:35.176437 (+ 2880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.176444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.176544 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.176629 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:35.182974 (+ 6345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.184113 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.186355 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.187118 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.187233 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.193545 (+ 6312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:35.193554 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:35.193560 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:35.193613 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:35.198330 (+ 4717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.199692 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.199699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.199703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.199886 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.200456 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.200464 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.200471 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.200882 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:35.202865 (+ 1983us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.203040 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.203328 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:20:35.207423 10884 ts_tablet_manager.cc:1918] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:35.207633 10884 log.cc:1196] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/5ec6485409d84f1ba1da62679616f7c9
I20260925 09:20:35.208312 10884 ts_tablet_manager.cc:1939] T 5ec6485409d84f1ba1da62679616f7c9 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Deleting consensus metadata
I20260925 09:20:35.219146 10885 ts_tablet_manager.cc:1918] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:35.219383 10885 log.cc:1196] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/5ec6485409d84f1ba1da62679616f7c9
I20260925 09:20:35.220098 10885 ts_tablet_manager.cc:1939] T 5ec6485409d84f1ba1da62679616f7c9 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Deleting consensus metadata
I20260925 09:20:35.225739 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.174179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.3.82.60:45463)
0925 09:20:35.174449 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.174459 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.202477 (+ 28018us) server_negotiation.cc:407] Connection header received
0925 09:20:35.202653 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.202658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.202753 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.202858 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:35.205033 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.206737 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.209423 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.210227 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.210359 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.216532 (+ 6173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:35.216540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:35.216545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:35.216603 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:35.220973 (+ 4370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.222205 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.222211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.222214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.222401 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.222957 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.222965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.222968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.223392 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:35.223700 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.225023 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.225305 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 09:20:35.228165 10883 ts_tablet_manager.cc:1918] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:35.228395 10883 log.cc:1196] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/5ec6485409d84f1ba1da62679616f7c9
I20260925 09:20:35.229223 10883 ts_tablet_manager.cc:1939] T 5ec6485409d84f1ba1da62679616f7c9 P 86845cbeecfb4beca94170ae87538a59: Deleting consensus metadata
I20260925 09:20:35.371868 9550 catalog_manager.cc:4841] TS 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029): tablet 5ec6485409d84f1ba1da62679616f7c9 (table test-table [id=08bf1270a83b40168322458a64ad8229]) successfully deleted
I20260925 09:20:35.372234 9550 catalog_manager.cc:4841] TS 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877): tablet 5ec6485409d84f1ba1da62679616f7c9 (table test-table [id=08bf1270a83b40168322458a64ad8229]) successfully deleted
I20260925 09:20:35.372503 9553 catalog_manager.cc:4841] TS 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133): tablet 5ec6485409d84f1ba1da62679616f7c9 (table test-table [id=08bf1270a83b40168322458a64ad8229]) successfully deleted
I20260925 09:20:35.372989 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 9534
I20260925 09:20:35.419277 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.402127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48454 (local address 127.3.82.60:45463)
0925 09:20:35.402369 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.402375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.402590 (+ 215us) server_negotiation.cc:407] Connection header received
0925 09:20:35.402847 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.402853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.402933 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.403066 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:35.404621 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.405760 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.407794 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.408433 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.408606 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.411725 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:35.411731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:35.411735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:35.411775 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:35.415890 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.416994 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.416999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.417002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.417169 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.417581 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.417588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.417591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.418021 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:35.418292 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.418671 (+ 379us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.418930 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:20:35.419426 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:35.400617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.3.82.61:41277)
0925 09:20:35.400820 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:35.400828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:35.402060 (+ 1232us) server_negotiation.cc:407] Connection header received
0925 09:20:35.402366 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:35.402372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:35.402482 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:35.402607 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:35.404937 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.406081 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.407475 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:35.408135 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:35.408270 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:35.412222 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:35.412233 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:35.412239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:35.412291 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:35.415890 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.416845 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.416851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.416855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.417056 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:35.417539 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:35.417548 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:35.417553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:35.418069 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:35.418335 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:35.418847 (+ 512us) server_negotiation.cc:299] Negotiation successful
0925 09:20:35.419082 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:20:36.129417 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.128386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:34731)
0925 09:20:36.128919 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.129063 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":293,"threads_started":1}
W20260925 09:20:36.129980 10826 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:36591 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.134305 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.133341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:39555)
0925 09:20:36.133965 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.134128 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":376,"threads_started":1}
W20260925 09:20:36.134852 10685 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:36591 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.137099 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.135890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:56917)
0925 09:20:36.136735 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.136894 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":578,"threads_started":1}
W20260925 09:20:36.137573 10537 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:36591 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.659334 10897 raft_consensus.cc:482] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02ef09e5832b484f871ed069a66f2770)
I20260925 09:20:36.659652 10897 raft_consensus.cc:504] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } }
I20260925 09:20:36.660334 10898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02ef09e5832b484f871ed069a66f2770)
I20260925 09:20:36.660641 10898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } }
I20260925 09:20:36.661486 10897 leader_election.cc:288] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591), 14d61c1451e84b708de00695bfe689f5 (127.3.82.61:41277)
I20260925 09:20:36.674481 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.661642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59072)
0925 09:20:36.673793 (+ 12151us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.674087 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11964,"thread_start_us":11726,"threads_started":1}
W20260925 09:20:36.675046 10343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.680619 10898 leader_election.cc:288] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591), c429f23af11e4acba958cf1dcde71bef (127.3.82.60:45463)
I20260925 09:20:36.684394 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.661741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41874)
0925 09:20:36.674191 (+ 12450us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.674217 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:36.674441 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:36.674999 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:36.675008 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:36.675046 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:36.675541 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.675557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.677741 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.677751 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.680782 (+ 3031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.680807 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.683263 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.683273 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.683552 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.683616 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 09:20:36.683729 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 09:20:36.683958 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12270,"thread_start_us":12291,"threads_started":1}
I20260925 09:20:36.685158 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.662007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.3.82.61:41277)
0925 09:20:36.673992 (+ 11985us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:36.674001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:36.674596 (+ 595us) server_negotiation.cc:407] Connection header received
0925 09:20:36.674695 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:36.674703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:36.674809 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:36.675745 (+ 936us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:36.676027 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.677501 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.681075 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.682982 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.684425 (+ 1443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.684554 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:36.684617 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 09:20:36.684773 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11831,"thread_start_us":12191,"threads_started":1}
I20260925 09:20:36.685206 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.674636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59086)
0925 09:20:36.684465 (+ 9829us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.684701 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9633,"thread_start_us":7568,"threads_started":1}
I20260925 09:20:36.685936 10184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c429f23af11e4acba958cf1dcde71bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "14d61c1451e84b708de00695bfe689f5" is_pre_election: true
I20260925 09:20:36.686380 10184 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c429f23af11e4acba958cf1dcde71bef in term 1.
I20260925 09:20:36.687047 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.686470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59090)
0925 09:20:36.686721 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.686842 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260925 09:20:36.687052 10341 leader_election.cc:302] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d61c1451e84b708de00695bfe689f5, c429f23af11e4acba958cf1dcde71bef; no voters:
W20260925 09:20:36.687345 10160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.687575 10897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260925 09:20:36.687485 10343 leader_election.cc:334] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591): Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.687748 10897 raft_consensus.cc:482] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02ef09e5832b484f871ed069a66f2770)
I20260925 09:20:36.687871 10897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:20:36.689306 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.688874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59100)
0925 09:20:36.689053 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.689147 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260925 09:20:36.689533 10160 leader_election.cc:334] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591): Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.690874 10897 raft_consensus.cc:504] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } }
I20260925 09:20:36.691830 10897 leader_election.cc:288] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [CANDIDATE]: Term 2 election: Requested vote from peers 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591), 14d61c1451e84b708de00695bfe689f5 (127.3.82.61:41277)
I20260925 09:20:36.692080 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.662475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:45463 (local address 127.0.0.1:48466)
0925 09:20:36.681819 (+ 19344us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.681878 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:36.682068 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:36.684781 (+ 2713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:36.684786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:36.684819 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:36.685210 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.685226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.687044 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.687052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.689460 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.689479 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.691117 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.691125 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.691337 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.691369 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:20:36.691497 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 09:20:36.691677 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19148,"thread_start_us":21597,"threads_started":1}
I20260925 09:20:36.692359 10184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c429f23af11e4acba958cf1dcde71bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "14d61c1451e84b708de00695bfe689f5"
I20260925 09:20:36.692559 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.691749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59114)
0925 09:20:36.691910 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.692099 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 09:20:36.692610 10184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:20:36.692653 10903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.663209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.3.82.60:45463)
0925 09:20:36.683087 (+ 19878us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:36.683097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:36.683127 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:20:36.683224 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:36.683233 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:36.683347 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:36.683492 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:36.685517 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.686889 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.689744 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.690890 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.691130 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.691265 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:36.691526 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 09:20:36.691676 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":19684,"thread_start_us":19464,"threads_started":1}
I20260925 09:20:36.693326 10368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14d61c1451e84b708de00695bfe689f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c429f23af11e4acba958cf1dcde71bef" is_pre_election: true
I20260925 09:20:36.693868 10368 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 14d61c1451e84b708de00695bfe689f5 in current term 2: Already voted for candidate c429f23af11e4acba958cf1dcde71bef in this term.
I20260925 09:20:36.694424 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.693918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59126)
0925 09:20:36.694163 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.694256 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260925 09:20:36.694646 10343 leader_election.cc:334] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591): Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
I20260925 09:20:36.694667 10158 leader_election.cc:302] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 14d61c1451e84b708de00695bfe689f5; no voters: 02ef09e5832b484f871ed069a66f2770, c429f23af11e4acba958cf1dcde71bef
I20260925 09:20:36.695705 10184 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c429f23af11e4acba958cf1dcde71bef in term 2.
I20260925 09:20:36.695991 10898 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:20:36.696173 10341 leader_election.cc:302] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 14d61c1451e84b708de00695bfe689f5, c429f23af11e4acba958cf1dcde71bef; no voters: 02ef09e5832b484f871ed069a66f2770
I20260925 09:20:36.696516 10897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:20:36.697062 10897 raft_consensus.cc:686] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [term 2 LEADER]: Becoming Leader. State: Replica: c429f23af11e4acba958cf1dcde71bef, State: Running, Role: LEADER
I20260925 09:20:36.697500 10897 consensus_queue.cc:237] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } }
I20260925 09:20:36.699690 10907 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: SysCatalogTable state changed. Reason: New leader c429f23af11e4acba958cf1dcde71bef. Latest consensus state: current_term: 2 leader_uuid: "c429f23af11e4acba958cf1dcde71bef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:36.700160 10907 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:36.700814 10909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:36.704485 10909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:36.705351 10909 catalog_manager.cc:1259] Loaded cluster ID: 15326be2d0954770a1c5730c683877ee
I20260925 09:20:36.705467 10909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:36.706724 10909 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:36.707844 10909 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Loaded TSK: 0
I20260925 09:20:36.708793 10909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:20:36.764725 10358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:36.772607 10358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:36.777361 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.776870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59148)
0925 09:20:36.777092 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.777188 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 09:20:36.777347 10184 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [term 2 FOLLOWER]: Refusing update from remote peer c429f23af11e4acba958cf1dcde71bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260925 09:20:36.778285 10907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260925 09:20:36.778448 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.778092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59158)
0925 09:20:36.778241 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.778325 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 09:20:36.778659 10343 consensus_peers.cc:487] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef -> Peer 02ef09e5832b484f871ed069a66f2770 (127.3.82.62:36591): Couldn't send request to peer 02ef09e5832b484f871ed069a66f2770. Status: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:20:36.781138 10898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c429f23af11e4acba958cf1dcde71bef. Latest consensus state: current_term: 2 leader_uuid: "c429f23af11e4acba958cf1dcde71bef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:36.781519 10898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:20:36.783645 10897 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c429f23af11e4acba958cf1dcde71bef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:36.784021 10897 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:36.784741 10898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c429f23af11e4acba958cf1dcde71bef" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "02ef09e5832b484f871ed069a66f2770" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36591 } } peers { permanent_uuid: "14d61c1451e84b708de00695bfe689f5" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 41277 } attrs { promote: false } } peers { permanent_uuid: "c429f23af11e4acba958cf1dcde71bef" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 45463 } attrs { promote: false } } }
I20260925 09:20:36.785060 10898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d61c1451e84b708de00695bfe689f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:20:36.803794 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.794061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:36029 (local address 127.0.0.1:53774)
0925 09:20:36.794233 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.794254 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:36.794389 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:36.794989 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:36.794994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:36.795021 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:36.795304 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.795320 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.797574 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.797583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.800434 (+ 2851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.800459 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.802540 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.802550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.802894 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.802977 (+ 83us) client_negotiation.cc:769] Sending connection context
0925 09:20:36.803070 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 09:20:36.803303 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 09:20:36.806296 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.794168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.3.82.1:36029)
0925 09:20:36.794638 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:36.794643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:36.794659 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:20:36.794717 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:36.794720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:36.794821 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:36.795724 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:36.795902 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.797368 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.800838 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.802311 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.805467 (+ 3156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.805630 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:36.805701 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 09:20:36.805912 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":272,"threads_started":1}
I20260925 09:20:36.808449 10472 tablet_service.cc:1467] Processing CreateTablet for tablet f6a528cdf61549358d801926c1147c76 (DEFAULT_TABLE table=test-table [id=b2a23806c17f45b890ad2179b7e23608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:36.809012 10472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a528cdf61549358d801926c1147c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:36.814419 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.794865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:39133 (local address 127.0.0.1:51980)
0925 09:20:36.795256 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.795277 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:36.795421 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:36.807758 (+ 12337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:36.807763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:36.807790 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:36.808066 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.808078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.809373 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.809381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.811407 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.811422 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.813391 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.813397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.813624 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.813692 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 09:20:36.813787 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 09:20:36.814020 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260925 09:20:36.814656 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.796038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:44877 (local address 127.0.0.1:47476)
0925 09:20:36.804161 (+ 8123us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.804189 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:36.804336 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:36.808466 (+ 4130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:36.808470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:36.808511 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:36.808719 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.808729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.810146 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.810151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.812038 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:36.812053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.813748 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.813756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:36.814017 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.814047 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:20:36.814147 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 09:20:36.814291 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7951,"thread_start_us":11420,"threads_started":1}
I20260925 09:20:36.815105 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.794989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.3.82.2:44877)
0925 09:20:36.808148 (+ 13159us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:36.808155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:36.808175 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:20:36.808244 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:36.808249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:36.808319 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:36.808427 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:36.808857 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.809999 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.812219 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.813517 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.813731 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.813891 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:36.814193 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 09:20:36.814349 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13030,"thread_start_us":12937,"threads_started":1}
I20260925 09:20:36.815356 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.795691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51980 (local address 127.3.82.3:39133)
0925 09:20:36.807425 (+ 11734us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:36.807434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:36.807456 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:20:36.807534 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:36.807538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:36.807604 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:36.807736 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:36.808243 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.809236 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.812220 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:36.813182 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:36.813332 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:36.813466 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:36.814796 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0925 09:20:36.814968 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11603,"thread_start_us":11479,"threads_started":1}
I20260925 09:20:36.815939 10620 tablet_service.cc:1467] Processing CreateTablet for tablet f6a528cdf61549358d801926c1147c76 (DEFAULT_TABLE table=test-table [id=b2a23806c17f45b890ad2179b7e23608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:36.816098 10761 tablet_service.cc:1467] Processing CreateTablet for tablet f6a528cdf61549358d801926c1147c76 (DEFAULT_TABLE table=test-table [id=b2a23806c17f45b890ad2179b7e23608]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:36.816526 10761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a528cdf61549358d801926c1147c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:36.816569 10620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a528cdf61549358d801926c1147c76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:36.821623 10921 tablet_bootstrap.cc:492] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Bootstrap starting.
I20260925 09:20:36.823411 10921 tablet_bootstrap.cc:654] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:36.823712 10920 tablet_bootstrap.cc:492] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Bootstrap starting.
I20260925 09:20:36.825706 10923 tablet_bootstrap.cc:492] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Bootstrap starting.
I20260925 09:20:36.826324 10920 tablet_bootstrap.cc:654] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:36.826946 10921 tablet_bootstrap.cc:492] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: No bootstrap required, opened a new log
I20260925 09:20:36.827106 10921 ts_tablet_manager.cc:1397] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 09:20:36.827666 10923 tablet_bootstrap.cc:654] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:36.828040 10921 raft_consensus.cc:348] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:36.828297 10921 raft_consensus.cc:374] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:36.828399 10921 raft_consensus.cc:729] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86845cbeecfb4beca94170ae87538a59, State: Initialized, Role: FOLLOWER
I20260925 09:20:36.828732 10921 consensus_queue.cc:260] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:36.829627 10921 ts_tablet_manager.cc:1428] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 09:20:36.830168 10923 tablet_bootstrap.cc:492] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: No bootstrap required, opened a new log
I20260925 09:20:36.830163 10920 tablet_bootstrap.cc:492] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: No bootstrap required, opened a new log
I20260925 09:20:36.830314 10923 ts_tablet_manager.cc:1397] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:20:36.830315 10920 ts_tablet_manager.cc:1397] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 09:20:36.831117 10920 raft_consensus.cc:348] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:36.831354 10920 raft_consensus.cc:374] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:36.831449 10920 raft_consensus.cc:729] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1ab20af64f4c5e8b8d12e4cdfb8880, State: Initialized, Role: FOLLOWER
I20260925 09:20:36.831128 10923 raft_consensus.cc:348] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:36.831838 10923 raft_consensus.cc:374] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:36.831707 10920 consensus_queue.cc:260] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:36.831934 10923 raft_consensus.cc:729] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac4bcf51fec4ac3ad9018d16ff2f6fa, State: Initialized, Role: FOLLOWER
I20260925 09:20:36.832247 10923 consensus_queue.cc:260] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:36.836778 10827 heartbeater.cc:503] Master 127.3.82.60:45463 was elected leader, sending a full tablet report...
I20260925 09:20:36.844156 10920 ts_tablet_manager.cc:1428] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Time spent starting tablet: real 0.014s user 0.004s sys 0.006s
I20260925 09:20:36.844522 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.830394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:57009)
0925 09:20:36.843898 (+ 13504us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.844108 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13302,"thread_start_us":13080,"threads_started":1}
I20260925 09:20:36.846254 10923 ts_tablet_manager.cc:1428] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260925 09:20:36.848884 10686 heartbeater.cc:503] Master 127.3.82.60:45463 was elected leader, sending a full tablet report...
I20260925 09:20:36.849406 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.845347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:52337)
0925 09:20:36.848872 (+ 3525us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.849139 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3344,"threads_started":1}
I20260925 09:20:36.851042 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.844000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:47223)
0925 09:20:36.850583 (+ 6583us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.850741 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6395,"thread_start_us":6247,"threads_started":1}
I20260925 09:20:36.851344 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.845632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:54897)
0925 09:20:36.850910 (+ 5278us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.851061 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5148,"threads_started":1}
I20260925 09:20:36.852730 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.852283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:36689)
0925 09:20:36.852486 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.852575 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 09:20:36.854818 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:36.854384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:34399)
0925 09:20:36.854581 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:36.854680 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 09:20:36.855122 10538 heartbeater.cc:503] Master 127.3.82.60:45463 was elected leader, sending a full tablet report...
W20260925 09:20:36.969516 10688 tablet.cc:2369] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:37.176506 10928 raft_consensus.cc:482] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:37.177011 10928 raft_consensus.cc:504] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:37.178819 10928 leader_election.cc:288] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133), 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877)
I20260925 09:20:37.186157 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.179360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:34683 (local address 127.3.82.3:39133)
0925 09:20:37.179564 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:37.179572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:37.180210 (+ 638us) server_negotiation.cc:407] Connection header received
0925 09:20:37.180349 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:37.180355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:37.180448 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:37.180530 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:37.181356 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.182371 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.184315 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.185109 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.185260 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.185364 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:37.185709 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 09:20:37.185865 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:20:37.186275 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.179342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:39133 (local address 127.3.82.1:34683)
0925 09:20:37.180100 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.180113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:37.180228 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:37.180546 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:37.180552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:37.180604 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:37.181112 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:37.181125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.182509 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.182518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:37.184140 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:37.184172 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.185344 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.185350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:37.185523 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.185573 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 09:20:37.185683 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:20:37.185823 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":24,"thread_start_us":485,"threads_started":1}
I20260925 09:20:37.186822 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.179346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:43367 (local address 127.3.82.2:44877)
0925 09:20:37.179793 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:37.179803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:37.179837 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:20:37.180001 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:37.180008 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:37.180098 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:37.180251 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:37.180997 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.182128 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.184161 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.184992 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.185187 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.185305 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:37.185871 (+ 566us) server_negotiation.cc:299] Negotiation successful
0925 09:20:37.186065 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260925 09:20:37.186872 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.179167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:44877 (local address 127.3.82.1:43367)
0925 09:20:37.179562 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.179582 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:37.179812 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:37.180285 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:37.180293 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:37.180320 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:37.180739 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:37.180780 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.182332 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.182338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:37.183922 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:37.183939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.185237 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.185244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:37.185582 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.185633 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 09:20:37.186453 (+ 820us) client_negotiation.cc:240] Negotiation successful
0925 09:20:37.186597 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1}
I20260925 09:20:37.187280 10781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a528cdf61549358d801926c1147c76" candidate_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86845cbeecfb4beca94170ae87538a59" is_pre_election: true
I20260925 09:20:37.187801 10781 raft_consensus.cc:2454] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1ab20af64f4c5e8b8d12e4cdfb8880 in term 0.
I20260925 09:20:37.187801 10640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a528cdf61549358d801926c1147c76" candidate_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" is_pre_election: true
I20260925 09:20:37.188045 10640 raft_consensus.cc:2454] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1ab20af64f4c5e8b8d12e4cdfb8880 in term 0.
I20260925 09:20:37.188522 10428 leader_election.cc:302] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1ab20af64f4c5e8b8d12e4cdfb8880, 86845cbeecfb4beca94170ae87538a59; no voters:
I20260925 09:20:37.188985 10928 raft_consensus.cc:2781] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:37.189113 10928 raft_consensus.cc:482] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:37.189189 10928 raft_consensus.cc:3037] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:37.191574 10928 raft_consensus.cc:504] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:37.192169 10928 leader_election.cc:288] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [CANDIDATE]: Term 1 election: Requested vote from peers 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133), 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877)
I20260925 09:20:37.192845 10781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a528cdf61549358d801926c1147c76" candidate_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86845cbeecfb4beca94170ae87538a59"
I20260925 09:20:37.192941 10640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a528cdf61549358d801926c1147c76" candidate_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa"
I20260925 09:20:37.193087 10781 raft_consensus.cc:3037] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:37.193224 10640 raft_consensus.cc:3037] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:37.195572 10640 raft_consensus.cc:2454] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1ab20af64f4c5e8b8d12e4cdfb8880 in term 1.
I20260925 09:20:37.195986 10781 raft_consensus.cc:2454] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1ab20af64f4c5e8b8d12e4cdfb8880 in term 1.
I20260925 09:20:37.196139 10425 leader_election.cc:302] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac4bcf51fec4ac3ad9018d16ff2f6fa, 4f1ab20af64f4c5e8b8d12e4cdfb8880; no voters:
I20260925 09:20:37.196520 10928 raft_consensus.cc:2781] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:37.197045 10928 raft_consensus.cc:686] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1ab20af64f4c5e8b8d12e4cdfb8880, State: Running, Role: LEADER
I20260925 09:20:37.198019 10928 consensus_queue.cc:237] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } }
I20260925 09:20:37.198340 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.197800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:34361)
0925 09:20:37.198018 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.198151 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 09:20:37.201793 10358 catalog_manager.cc:5495] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1ab20af64f4c5e8b8d12e4cdfb8880" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 } health_report { overall_health: UNKNOWN } } }
I20260925 09:20:37.283883 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.277754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.3.82.1:36029)
0925 09:20:37.278043 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:37.278050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:37.278230 (+ 180us) server_negotiation.cc:407] Connection header received
0925 09:20:37.278591 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:37.278595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:37.278678 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:37.278847 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:37.279254 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.279999 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.281587 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.282044 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.282203 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.282684 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:37.283251 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:37.283413 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:37.283517 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 09:20:37.283606 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260925 09:20:37.289011 10640 raft_consensus.cc:1261] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Refusing update from remote peer 4f1ab20af64f4c5e8b8d12e4cdfb8880: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:37.289077 10781 raft_consensus.cc:1261] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Refusing update from remote peer 4f1ab20af64f4c5e8b8d12e4cdfb8880: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:37.290000 10934 consensus_queue.cc:1035] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86845cbeecfb4beca94170ae87538a59" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:37.290392 10928 consensus_queue.cc:1035] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ac4bcf51fec4ac3ad9018d16ff2f6fa" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:37.293047 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.292298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:48929)
0925 09:20:37.292650 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.292769 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260925 09:20:37.295465 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.294827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:47181)
0925 09:20:37.295027 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.295203 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 09:20:37.298938 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.298456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59174)
0925 09:20:37.298655 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.298755 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260925 09:20:37.301074 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.300258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:33515)
0925 09:20:37.300743 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.300857 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332}
I20260925 09:20:37.303319 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.302906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59190)
0925 09:20:37.303057 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.303152 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 09:20:37.307484 10941 mvcc.cc:205] Tried to move back new op lower bound from 7333183640726245376 to 7333183640367734784. Current Snapshot: MvccSnapshot[applied={T|T < 7333183640726245376}]
I20260925 09:20:37.308454 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.307661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.3:46831)
0925 09:20:37.308165 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.308271 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390}
I20260925 09:20:37.308588 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.307947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.2:49975)
0925 09:20:37.308251 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.308363 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260925 09:20:37.308806 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.308349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.3.82.1:41173)
0925 09:20:37.308573 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.308676 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260925 09:20:37.333012 10358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:37.333345 10358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:37.340047 10358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef: Sending DeleteTablet for 3 replicas of tablet f6a528cdf61549358d801926c1147c76
I20260925 09:20:37.341216 10472 tablet_service.cc:1514] Processing DeleteTablet for tablet f6a528cdf61549358d801926c1147c76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:37 UTC) from {username='oryx', principal='oryx/127.3.82.60@KRBTEST.COM'} at 127.0.0.1:53774
I20260925 09:20:37.341645 10761 tablet_service.cc:1514] Processing DeleteTablet for tablet f6a528cdf61549358d801926c1147c76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:37 UTC) from {username='oryx', principal='oryx/127.3.82.60@KRBTEST.COM'} at 127.0.0.1:51980
I20260925 09:20:37.343240 10620 tablet_service.cc:1514] Processing DeleteTablet for tablet f6a528cdf61549358d801926c1147c76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:37 UTC) from {username='oryx', principal='oryx/127.3.82.60@KRBTEST.COM'} at 127.0.0.1:47476
I20260925 09:20:37.343971 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 10345
I20260925 09:20:37.354252 10950 tablet_replica.cc:330] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: stopping tablet replica
I20260925 09:20:37.354777 10950 raft_consensus.cc:2229] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:37.355286 10950 raft_consensus.cc:2258] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:37.356786 10950 ts_tablet_manager.cc:1905] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:37.358644 10951 tablet_replica.cc:330] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: stopping tablet replica
I20260925 09:20:37.358870 10951 raft_consensus.cc:2229] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:37.359155 10951 raft_consensus.cc:2258] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:37.360406 10951 ts_tablet_manager.cc:1905] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:37.367295 10951 ts_tablet_manager.cc:1918] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:37.367295 10950 ts_tablet_manager.cc:1918] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:37.367457 10951 log.cc:1196] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/f6a528cdf61549358d801926c1147c76
I20260925 09:20:37.367937 10951 ts_tablet_manager.cc:1939] T f6a528cdf61549358d801926c1147c76 P 1ac4bcf51fec4ac3ad9018d16ff2f6fa: Deleting consensus metadata
I20260925 09:20:37.368924 10949 tablet_replica.cc:330] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: stopping tablet replica
I20260925 09:20:37.369325 10949 raft_consensus.cc:2229] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:37.369360 10341 catalog_manager.cc:4841] TS 1ac4bcf51fec4ac3ad9018d16ff2f6fa (127.3.82.2:44877): tablet f6a528cdf61549358d801926c1147c76 (table test-table [id=b2a23806c17f45b890ad2179b7e23608]) successfully deleted
I20260925 09:20:37.370005 10950 log.cc:1196] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/f6a528cdf61549358d801926c1147c76
I20260925 09:20:37.370185 10949 raft_consensus.cc:2258] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:37.370564 10950 ts_tablet_manager.cc:1939] T f6a528cdf61549358d801926c1147c76 P 86845cbeecfb4beca94170ae87538a59: Deleting consensus metadata
I20260925 09:20:37.371793 10344 catalog_manager.cc:4841] TS 86845cbeecfb4beca94170ae87538a59 (127.3.82.3:39133): tablet f6a528cdf61549358d801926c1147c76 (table test-table [id=b2a23806c17f45b890ad2179b7e23608]) successfully deleted
I20260925 09:20:37.379307 10949 ts_tablet_manager.cc:1905] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:37.387223 10949 ts_tablet_manager.cc:1918] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:37.387856 10949 log.cc:1196] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/f6a528cdf61549358d801926c1147c76
I20260925 09:20:37.388049 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.344949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.3.82.1:36029)
0925 09:20:37.345278 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:37.345285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:37.357745 (+ 12460us) server_negotiation.cc:407] Connection header received
0925 09:20:37.358167 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:37.358177 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:37.358292 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:37.358381 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:37.369138 (+ 10757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.370454 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.373000 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.373700 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.373803 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.377584 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:37.377605 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:37.377626 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:37.377676 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:37.382855 (+ 5179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:37.383912 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:37.383922 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:37.383925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:37.384133 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:37.384609 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:37.384614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:37.384618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:37.385158 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:37.385636 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:37.386925 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0925 09:20:37.387165 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260925 09:20:37.388931 10949 ts_tablet_manager.cc:1939] T f6a528cdf61549358d801926c1147c76 P 4f1ab20af64f4c5e8b8d12e4cdfb8880: Deleting consensus metadata
I20260925 09:20:37.388960 10532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:37.493813 10341 catalog_manager.cc:4841] TS 4f1ab20af64f4c5e8b8d12e4cdfb8880 (127.3.82.1:36029): tablet f6a528cdf61549358d801926c1147c76 (table test-table [id=b2a23806c17f45b890ad2179b7e23608]) successfully deleted
I20260925 09:20:37.493970 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 10345
I20260925 09:20:37.512959 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 10549
I20260925 09:20:37.529394 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.514103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.3.82.2:44877)
0925 09:20:37.514313 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:37.514320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:37.514343 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:20:37.514647 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:37.514652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:37.514730 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:37.514821 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:37.516404 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.517195 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.518885 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.519302 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.519507 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.522339 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:37.522359 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:37.522375 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:37.522416 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:37.526188 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:37.527046 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:37.527054 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:37.527057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:37.527237 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:37.527800 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:37.527805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:37.527808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:37.528267 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:37.528615 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:37.528860 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 09:20:37.529060 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:20:37.530298 10680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:37.628937 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 10549
I20260925 09:20:37.647178 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 10698
I20260925 09:20:37.663323 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.648049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.3.82.3:39133)
0925 09:20:37.648240 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:37.648244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:37.648263 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:20:37.648371 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:37.648375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:37.648544 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:37.648637 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:37.650199 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.651089 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.652541 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:37.653027 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:37.653156 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:37.656420 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:37.656440 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:37.656456 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:37.656514 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:37.659717 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:37.660616 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:37.660633 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:37.660635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:37.660809 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:37.661257 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:37.661264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:37.661267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:37.661703 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:37.662204 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:37.662675 (+ 471us) server_negotiation.cc:299] Negotiation successful
0925 09:20:37.662916 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 09:20:37.664036 10821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:37.775368 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.774827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59200)
0925 09:20:37.775042 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.775172 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260925 09:20:37.776611 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.776174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36591 (local address 127.0.0.1:59204)
0925 09:20:37.776345 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.776433 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:36591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:20:37.790030 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 10698
I20260925 09:20:37.812776 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 10029
I20260925 09:20:37.814332 10204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:37.936754 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 10029
I20260925 09:20:37.954880 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 10130
I20260925 09:20:37.955663 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:37.954996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:41277 (local address 127.0.0.1:41878)
0925 09:20:37.955271 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:37.955409 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:41277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260925 09:20:37.955927 10341 consensus_peers.cc:487] T 00000000000000000000000000000000 P c429f23af11e4acba958cf1dcde71bef -> Peer 14d61c1451e84b708de00695bfe689f5 (127.3.82.61:41277): Couldn't send request to peer 14d61c1451e84b708de00695bfe689f5. Status: Network error: Client connection negotiation failed: client connection to 127.3.82.61:41277: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:20:37.956300 10388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:38.083256 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 10130
2026-09-25T09:20:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10369 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:38 dist-test-slave-4wm5 krb5kdc[10960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:38 dist-test-slave-4wm5 krb5kdc[10960](info): set up 2 sockets
Sep 25 09:20:38 dist-test-slave-4wm5 krb5kdc[10960](info): commencing operation
krb5kdc: starting...
W20260925 09:20:39.222952 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:39 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab.
Entry for principal oryx/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab.
I20260925 09:20:39.462339 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45477
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab
--principal=oryx/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:39.806252 10973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:39.806641 10973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:39.806802 10973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:39.814374 10973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:39.814538 10973 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 09:20:39.814652 10973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:39.814769 10973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:39.814857 10973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:39.814949 10973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:39.825407 10973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45477
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.62.keytab
--principal=oryx/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:39.826927 10973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:39.828377 10973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:39.842127 10981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:39.842178 10979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:39.842203 10978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:39.843063 10973 server_base.cc:1034] running on GCE node
I20260925 09:20:39.843955 10973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:39.845690 10973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:39.846995 10973 hybrid_clock.cc:648] HybridClock initialized: now 1790328039846924 us; error 54 us; skew 500 ppm
I20260925 09:20:39.847420 10973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:39 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:39.855888 10973 init.cc:377] Logged in from keytab as oryx/127.3.82.62@KRBTEST.COM (short username oryx)
I20260925 09:20:39.857713 10973 webserver.cc:460] Webserver started at http://127.3.82.62:35819/ using document root <none> and password file <none>
I20260925 09:20:39.858566 10973 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:39.858668 10973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:39.858973 10973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:39.862385 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "9a83bcb6b82a4086a1844e77da1b999e"
format_stamp: "Formatted at 2026-09-25 09:20:39 on dist-test-slave-4wm5"
server_key: "675cf53c50afff2bac8e6ae9da2b2e31"
server_key_iv: "cbdad0939758b74b74613015c308babc"
server_key_version: "encryptionkey@0"
I20260925 09:20:39.863231 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "9a83bcb6b82a4086a1844e77da1b999e"
format_stamp: "Formatted at 2026-09-25 09:20:39 on dist-test-slave-4wm5"
server_key: "675cf53c50afff2bac8e6ae9da2b2e31"
server_key_iv: "cbdad0939758b74b74613015c308babc"
server_key_version: "encryptionkey@0"
I20260925 09:20:39.868921 10973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 09:20:39.872900 10988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:39.874681 10973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:39.874853 10973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "9a83bcb6b82a4086a1844e77da1b999e"
format_stamp: "Formatted at 2026-09-25 09:20:39 on dist-test-slave-4wm5"
server_key: "675cf53c50afff2bac8e6ae9da2b2e31"
server_key_iv: "cbdad0939758b74b74613015c308babc"
server_key_version: "encryptionkey@0"
I20260925 09:20:39.875069 10973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:39.903093 10973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:39.907998 10973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:39.908346 10973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:39.932255 10973 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45477
I20260925 09:20:39.932314 11049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45477 every 8 connection(s)
I20260925 09:20:39.933736 10973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:39.937018 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 10973
I20260925 09:20:39.937237 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:39.938133 3400 external_mini_cluster.cc:1428] Setting key 4d76df167a85d50186a440c3f001041b
I20260925 09:20:39.938405 11050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:39.953097 11050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Bootstrap starting.
Sep 25 09:20:39 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:39.956740 11050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:39.958154 11050 log.cc:824] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:39.961247 11050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: No bootstrap required, opened a new log
I20260925 09:20:39.963960 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:39.940504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.3.82.62:45477)
0925 09:20:39.941382 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:39.941395 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:39.941443 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:20:39.943930 (+ 2487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:39.943961 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:39.944511 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:39.945052 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:39.946907 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:39.948328 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:39.949765 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:39.950726 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:39.950946 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:39.955545 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:39.955555 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:39.955571 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:39.955619 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:39.959213 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:39.960177 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:39.960186 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:39.960193 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:39.960388 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:39.960809 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:39.960814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:39.960819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:39.961436 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:39.961769 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:39.962301 (+ 532us) server_negotiation.cc:299] Negotiation successful
0925 09:20:39.962794 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":380,"threads_started":1}
I20260925 09:20:39.970829 11050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a83bcb6b82a4086a1844e77da1b999e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45477 } }
I20260925 09:20:39.971277 11050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:39.971391 11050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a83bcb6b82a4086a1844e77da1b999e, State: Initialized, Role: FOLLOWER
I20260925 09:20:39.972004 11050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a83bcb6b82a4086a1844e77da1b999e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45477 } }
I20260925 09:20:39.972286 11050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:39.972435 11050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:39.972607 11050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:39.975260 11050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a83bcb6b82a4086a1844e77da1b999e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45477 } }
I20260925 09:20:39.975842 11050 leader_election.cc:302] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a83bcb6b82a4086a1844e77da1b999e; no voters:
I20260925 09:20:39.976609 11050 leader_election.cc:288] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:39.976838 11055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:39.977557 11055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [term 1 LEADER]: Becoming Leader. State: Replica: 9a83bcb6b82a4086a1844e77da1b999e, State: Running, Role: LEADER
I20260925 09:20:39.978220 11055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a83bcb6b82a4086a1844e77da1b999e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45477 } }
I20260925 09:20:39.979244 11050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:39.981483 11056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a83bcb6b82a4086a1844e77da1b999e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a83bcb6b82a4086a1844e77da1b999e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45477 } } }
I20260925 09:20:39.981989 11056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:39.982468 11057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a83bcb6b82a4086a1844e77da1b999e. Latest consensus state: current_term: 1 leader_uuid: "9a83bcb6b82a4086a1844e77da1b999e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a83bcb6b82a4086a1844e77da1b999e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45477 } } }
I20260925 09:20:39.982758 11057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:39.982991 11063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:39.990204 11063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:40.002367 11073 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:20:40.004359 11063 catalog_manager.cc:1347] Generated new cluster ID: a04fcb6d56304c15abd28830c5e09e88
I20260925 09:20:40.004491 11063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:40.020351 11063 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:40.022006 11063 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:40.033149 11063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Generated new TSK 0
I20260925 09:20:40.034133 11063 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab.
Entry for principal oryx/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab.
I20260925 09:20:40.102118 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45477
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:40.469321 11076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:40.469655 11076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:40.469875 11076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:40.477645 11076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:40.477780 11076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:40.477977 11076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:20:40.478116 11076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:40.489049 11076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:45477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:40.490552 11076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:40.491945 11076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:40.506577 11081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:40.506654 11084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:40.506676 11082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:40.507609 11076 server_base.cc:1034] running on GCE node
I20260925 09:20:40.508275 11076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:40.509251 11076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:40.510479 11076 hybrid_clock.cc:648] HybridClock initialized: now 1790328040510410 us; error 40 us; skew 500 ppm
I20260925 09:20:40.510852 11076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:40 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328040, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:40.514379 11076 init.cc:377] Logged in from keytab as oryx/127.3.82.1@KRBTEST.COM (short username oryx)
I20260925 09:20:40.516026 11076 webserver.cc:460] Webserver started at http://127.3.82.1:44123/ using document root <none> and password file <none>
I20260925 09:20:40.516806 11076 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:40.516948 11076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:40.517297 11076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:40.520560 11076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "4508eaa827384d0cbf0150dda973a447"
format_stamp: "Formatted at 2026-09-25 09:20:40 on dist-test-slave-4wm5"
server_key: "989a2507884b00d5befb02134becef51"
server_key_iv: "1e0c9ec226481f145841e4b0b5603dd2"
server_key_version: "encryptionkey@0"
I20260925 09:20:40.521469 11076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "4508eaa827384d0cbf0150dda973a447"
format_stamp: "Formatted at 2026-09-25 09:20:40 on dist-test-slave-4wm5"
server_key: "989a2507884b00d5befb02134becef51"
server_key_iv: "1e0c9ec226481f145841e4b0b5603dd2"
server_key_version: "encryptionkey@0"
I20260925 09:20:40.527247 11076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 09:20:40.531221 11091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:40.532796 11076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:20:40.533000 11076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "4508eaa827384d0cbf0150dda973a447"
format_stamp: "Formatted at 2026-09-25 09:20:40 on dist-test-slave-4wm5"
server_key: "989a2507884b00d5befb02134becef51"
server_key_iv: "1e0c9ec226481f145841e4b0b5603dd2"
server_key_version: "encryptionkey@0"
I20260925 09:20:40.533298 11076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:40.555922 11076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:40.560530 11076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:40.560786 11076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:40.563782 11076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:40.563900 11076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:40.564024 11076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:40.564108 11076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:20:40 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328040, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:40.592303 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:40.564468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:56587 (local address 127.3.82.62:45477)
0925 09:20:40.565147 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:40.565152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:40.566423 (+ 1271us) server_negotiation.cc:407] Connection header received
0925 09:20:40.568618 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:40.568625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:40.568714 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:40.568899 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:40.571436 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.572566 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.575302 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.575867 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.576059 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:40.582434 (+ 6375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:40.582441 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:40.582444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:40.582483 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:40.586556 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:40.587806 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:40.587811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:40.587815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:40.587992 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:40.588680 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:40.588688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:40.588693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:40.589102 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:40.589266 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:40.591787 (+ 2521us) server_negotiation.cc:299] Negotiation successful
0925 09:20:40.592061 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":424,"threads_started":1}
I20260925 09:20:40.592873 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:40.565236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.3.82.1:56587)
0925 09:20:40.566239 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:40.566287 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:40.568228 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:40.569152 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:40.569168 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:40.570146 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:40.571104 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:40.571157 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.572761 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.572772 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:40.575129 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:40.575143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.576204 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.576219 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:40.576617 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:40.577684 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:40.577737 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:40.582058 (+ 4321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:40.586778 (+ 4720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:40.586794 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:40.586823 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:40.587637 (+ 814us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:40.588151 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:40.588159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:40.588165 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:40.588528 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:40.589244 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:40.589253 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:40.589904 (+ 651us) client_negotiation.cc:769] Sending connection context
0925 09:20:40.590345 (+ 441us) client_negotiation.cc:240] Negotiation successful
0925 09:20:40.591044 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":397,"threads_started":1}
I20260925 09:20:40.612812 11076 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:35583
I20260925 09:20:40.612874 11211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:35583 every 8 connection(s)
I20260925 09:20:40.614408 11076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:40.616950 11212 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45477
I20260925 09:20:40.617360 11212 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:40.618080 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:40.600214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.3.82.1:35079)
0925 09:20:40.601053 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:40.601077 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:40.601283 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:40.601970 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:40.601978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:40.602748 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:40.603192 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:40.603206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.604530 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.604543 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:40.607116 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:40.607137 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.608262 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.608272 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:40.608515 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:40.609367 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:40.609405 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:40.610901 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:40.614534 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:40.614540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:40.614543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:40.615166 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:40.615663 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:40.615671 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:40.615677 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:40.615890 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:40.616976 (+ 1086us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:40.616983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:40.617217 (+ 234us) client_negotiation.cc:769] Sending connection context
0925 09:20:40.617327 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:20:40.617670 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":451,"threads_started":1}
I20260925 09:20:40.618163 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:40.600159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:35079 (local address 127.3.82.62:45477)
0925 09:20:40.600398 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:40.600406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:40.601191 (+ 785us) server_negotiation.cc:407] Connection header received
0925 09:20:40.601469 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:40.601478 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:40.601605 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:40.601728 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:40.603387 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.604347 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.607380 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:40.608021 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:40.608144 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:40.611130 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:40.611135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:40.611138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:40.611175 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:40.614377 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:40.615343 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:40.615349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:40.615352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:40.615517 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:40.616070 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:40.616079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:40.616082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:40.616424 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:40.616604 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:40.617501 (+ 897us) server_negotiation.cc:299] Negotiation successful
0925 09:20:40.617776 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:20:40.618366 11212 heartbeater.cc:511] Master 127.3.82.62:45477 requested a full tablet report, sending...
I20260925 09:20:40.619915 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 11076
I20260925 09:20:40.620124 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:40.620855 11005 ts_manager.cc:194] Registered new tserver with Master: 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583)
I20260925 09:20:40.620965 3400 external_mini_cluster.cc:1428] Setting key b2b00f2da2612aff94d1283961c6c57b
I20260925 09:20:40.622985 11005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:56587
WARNING: no policy specified for oryx/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab.
Entry for principal oryx/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab.
I20260925 09:20:40.677951 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45477
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:41.032464 11216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:41.032835 11216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:41.033018 11216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:41.041116 11216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:41.041311 11216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:41.041463 11216 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:20:41.041601 11216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:41.052357 11216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:45477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:41.053804 11216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:41.055382 11216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:41.069922 11221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:41.069900 11224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:41.069943 11222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:41.070904 11216 server_base.cc:1034] running on GCE node
I20260925 09:20:41.071743 11216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:41.072831 11216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:41.074105 11216 hybrid_clock.cc:648] HybridClock initialized: now 1790328041074021 us; error 74 us; skew 500 ppm
I20260925 09:20:41.074534 11216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328041, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:41.078338 11216 init.cc:377] Logged in from keytab as oryx/127.3.82.2@KRBTEST.COM (short username oryx)
I20260925 09:20:41.080315 11216 webserver.cc:460] Webserver started at http://127.3.82.2:34563/ using document root <none> and password file <none>
I20260925 09:20:41.081132 11216 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:41.081265 11216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:41.081635 11216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:41.084823 11216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "32836bcc18a34ac784457e2922590f58"
format_stamp: "Formatted at 2026-09-25 09:20:41 on dist-test-slave-4wm5"
server_key: "d71f8fae201f3041dde1883dc8d18bee"
server_key_iv: "fe0452ba515b264c503f0acf33719c6d"
server_key_version: "encryptionkey@0"
I20260925 09:20:41.085762 11216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "32836bcc18a34ac784457e2922590f58"
format_stamp: "Formatted at 2026-09-25 09:20:41 on dist-test-slave-4wm5"
server_key: "d71f8fae201f3041dde1883dc8d18bee"
server_key_iv: "fe0452ba515b264c503f0acf33719c6d"
server_key_version: "encryptionkey@0"
I20260925 09:20:41.091876 11216 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260925 09:20:41.097034 11231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:41.098907 11216 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 09:20:41.099110 11216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "32836bcc18a34ac784457e2922590f58"
format_stamp: "Formatted at 2026-09-25 09:20:41 on dist-test-slave-4wm5"
server_key: "d71f8fae201f3041dde1883dc8d18bee"
server_key_iv: "fe0452ba515b264c503f0acf33719c6d"
server_key_version: "encryptionkey@0"
I20260925 09:20:41.099357 11216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:41.112402 11216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:41.117295 11216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:41.117596 11216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:41.120513 11216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:41.120641 11216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:41.120785 11216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:41.120865 11216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328041, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:41.152293 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.121484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:50157 (local address 127.3.82.62:45477)
0925 09:20:41.121940 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.121947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.123282 (+ 1335us) server_negotiation.cc:407] Connection header received
0925 09:20:41.126689 (+ 3407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.126697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.126809 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.126997 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:41.129771 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.131287 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.134295 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.135068 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.135233 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.141907 (+ 6674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:41.141917 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:41.141925 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:41.141984 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:41.146721 (+ 4737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.148182 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.148188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.148192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.148386 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.149144 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.149151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.149155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.149623 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:41.149805 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.151189 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.151638 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":205,"threads_started":1}
I20260925 09:20:41.153438 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.122188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.3.82.2:50157)
0925 09:20:41.123096 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.123151 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.126219 (+ 3068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.127283 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.127296 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.128312 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:41.129469 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.129497 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.131558 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.131566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.134091 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.134107 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.135495 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.135507 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.135895 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.136958 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:41.136988 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:41.141453 (+ 4465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:41.147005 (+ 5552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.147019 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.147057 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.147966 (+ 909us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.148582 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.148587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.148591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.148968 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.149798 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:41.149808 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:41.150515 (+ 707us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.150987 (+ 472us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.151645 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":354,"threads_started":1}
I20260925 09:20:41.178177 11314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.161149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.3.82.2:48793)
0925 09:20:41.161955 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.161972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.162152 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.162538 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.162543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.163042 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:41.163379 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.163390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.164434 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.164444 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.166315 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.166341 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.167634 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.167643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.167880 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.168748 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:41.168780 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:41.169716 (+ 936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:41.174865 (+ 5149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.174870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.174874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.175341 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.175865 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.175869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.175871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.176002 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.176991 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:41.176995 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:41.177178 (+ 183us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.177320 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.177637 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":441,"threads_started":1}
I20260925 09:20:41.178577 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.161232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:48793 (local address 127.3.82.62:45477)
0925 09:20:41.161542 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.161549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.162068 (+ 519us) server_negotiation.cc:407] Connection header received
0925 09:20:41.162288 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.162294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.162373 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.162522 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:41.163546 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.164284 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.166526 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.167349 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.167538 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.170089 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:41.170098 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:41.170105 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:41.170157 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:41.174639 (+ 4482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.175512 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.175522 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.175528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.175734 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.176133 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.176141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.176147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.176766 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:41.176994 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.177521 (+ 527us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.178059 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:20:41.183225 11216 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:43577
I20260925 09:20:41.183348 11351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:43577 every 8 connection(s)
I20260925 09:20:41.184500 11216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:41.184945 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 11216
I20260925 09:20:41.185151 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:41.185815 3400 external_mini_cluster.cc:1428] Setting key fd35a5840a351a6bf7cba217e2fba1c4
I20260925 09:20:41.187350 11352 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45477
I20260925 09:20:41.187624 11352 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:41.188373 11352 heartbeater.cc:511] Master 127.3.82.62:45477 requested a full tablet report, sending...
I20260925 09:20:41.190105 11004 ts_manager.cc:194] Registered new tserver with Master: 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577)
I20260925 09:20:41.191468 11004 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:50157
WARNING: no policy specified for oryx/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab.
Entry for principal oryx/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab.
I20260925 09:20:41.242877 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45477
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:41.595269 11356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:41.595741 11356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:41.595920 11356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:41.603582 11356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:41.603745 11356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:41.603901 11356 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 09:20:41.604050 11356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:41.614334 11356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.82.62:45477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:41.615741 11356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:41.617161 11356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:20:41.626827 11212 heartbeater.cc:503] Master 127.3.82.62:45477 was elected leader, sending a full tablet report...
W20260925 09:20:41.631788 11362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:41.631819 11364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:41.631762 11361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:41.632524 11356 server_base.cc:1034] running on GCE node
I20260925 09:20:41.633127 11356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:41.634233 11356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:41.635540 11356 hybrid_clock.cc:648] HybridClock initialized: now 1790328041635306 us; error 141 us; skew 500 ppm
I20260925 09:20:41.635913 11356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328041, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:41.639506 11356 init.cc:377] Logged in from keytab as oryx/127.3.82.3@KRBTEST.COM (short username oryx)
I20260925 09:20:41.641109 11356 webserver.cc:460] Webserver started at http://127.3.82.3:39199/ using document root <none> and password file <none>
I20260925 09:20:41.641812 11356 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:41.641981 11356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:41.642282 11356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:41.645396 11356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "89d9c87e3619401c9f907fb38e8fee3a"
format_stamp: "Formatted at 2026-09-25 09:20:41 on dist-test-slave-4wm5"
server_key: "23cc719c27658390aecd2f3036980670"
server_key_iv: "c0227a869a54fafaf6c0de4b0bc343b4"
server_key_version: "encryptionkey@0"
I20260925 09:20:41.646382 11356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "89d9c87e3619401c9f907fb38e8fee3a"
format_stamp: "Formatted at 2026-09-25 09:20:41 on dist-test-slave-4wm5"
server_key: "23cc719c27658390aecd2f3036980670"
server_key_iv: "c0227a869a54fafaf6c0de4b0bc343b4"
server_key_version: "encryptionkey@0"
I20260925 09:20:41.652277 11356 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260925 09:20:41.656528 11371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:41.658366 11356 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:20:41.658607 11356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "89d9c87e3619401c9f907fb38e8fee3a"
format_stamp: "Formatted at 2026-09-25 09:20:41 on dist-test-slave-4wm5"
server_key: "23cc719c27658390aecd2f3036980670"
server_key_iv: "c0227a869a54fafaf6c0de4b0bc343b4"
server_key_version: "encryptionkey@0"
I20260925 09:20:41.658883 11356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:41.685101 11356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:41.690135 11356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:41.690441 11356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:41.693383 11356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:41.693504 11356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:41.693637 11356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:41.693714 11356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328041, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:41.720252 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.694035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:56089 (local address 127.3.82.62:45477)
0925 09:20:41.696921 (+ 2886us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.696929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.696956 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:20:41.698907 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.698913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.699003 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.699140 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:41.701725 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.702887 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.705408 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.706047 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.706218 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.711253 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:41.711264 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:41.711272 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:41.711326 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:41.715919 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.717410 (+ 1491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.717420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.717514 (+ 94us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.717713 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.718271 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.718278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.718282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.718600 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:41.718763 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.719589 (+ 826us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.719842 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2747,"thread_start_us":2641,"threads_started":1}
I20260925 09:20:41.721776 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.694853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.3.82.3:56089)
0925 09:20:41.696305 (+ 1452us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.696382 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.698580 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.699460 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.699478 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.700391 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:41.701504 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.701538 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.703065 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.703083 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.705178 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.705199 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.706352 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.706362 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.706600 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.707337 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:41.707361 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:41.710898 (+ 3537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:41.716216 (+ 5318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.716232 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.716263 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.717240 (+ 977us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.717899 (+ 659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.717904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.717908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.718145 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.718774 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:41.718785 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:41.719194 (+ 409us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.719484 (+ 290us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.720079 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":474,"threads_started":1}
I20260925 09:20:41.746631 11438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.728680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.3.82.3:44767)
0925 09:20:41.730524 (+ 1844us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.730550 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.730782 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.731211 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.731215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.731763 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:41.732070 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.732081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.733231 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.733239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.734580 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.734605 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.735662 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.735673 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.735855 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.736933 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:41.736956 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:41.738574 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:41.743333 (+ 4759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.743343 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.743351 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.744154 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.744619 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:41.744625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:41.744632 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:41.744784 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:41.745356 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:41.745361 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:41.745577 (+ 216us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.745791 (+ 214us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.746212 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1629,"thread_start_us":1441,"threads_started":1}
I20260925 09:20:41.746960 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.728723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:44767 (local address 127.3.82.62:45477)
0925 09:20:41.728952 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.728961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.730680 (+ 1719us) server_negotiation.cc:407] Connection header received
0925 09:20:41.730945 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.730953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.731046 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.731177 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:41.732241 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.733082 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.734826 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.735437 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.735592 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.738914 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:41.738920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:41.738923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:41.738964 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:41.743106 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.744322 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.744331 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.744345 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.744485 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.744917 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.744928 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.744932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.745233 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:41.745396 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.745987 (+ 591us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.746353 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:20:41.760113 11356 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:44991
I20260925 09:20:41.760118 11491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:44991 every 8 connection(s)
I20260925 09:20:41.761631 11356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:41.764163 11492 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45477
I20260925 09:20:41.764427 11492 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:41.765116 11492 heartbeater.cc:511] Master 127.3.82.62:45477 requested a full tablet report, sending...
I20260925 09:20:41.766577 11004 ts_manager.cc:194] Registered new tserver with Master: 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:41.767410 11004 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:56089
I20260925 09:20:41.768962 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 11356
I20260925 09:20:41.769136 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:41.769683 3400 external_mini_cluster.cc:1428] Setting key 09e65bb60d4fa9ba84e7051a1cb22c5a
I20260925 09:20:41.772993 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328041, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:20:41.826879 11380 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:41.815670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.3.82.62:45477)
0925 09:20:41.815874 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.815878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.816098 (+ 220us) server_negotiation.cc:407] Connection header received
0925 09:20:41.816229 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.816234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.816312 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.816413 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:41.817813 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.818652 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.819962 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.820394 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.820516 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.826376 (+ 5860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50974: BlockingRecv error: recv got EOF from 127.0.0.1:50974 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 25 09:20:41 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328041, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:41.845541 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.830442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.3.82.62:45477)
0925 09:20:41.830708 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.830713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.830941 (+ 228us) server_negotiation.cc:407] Connection header received
0925 09:20:41.831097 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.831102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.831175 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.831282 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:41.832457 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.833274 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.834630 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.835107 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.835219 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.839347 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:41.839360 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:41.839364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:41.839413 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:41.842907 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.843720 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.843725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.843728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.843877 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:41.844264 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:41.844267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:41.844269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:41.844558 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:41.844809 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.845070 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.845280 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260925 09:20:41.850085 11004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:41.857365 11004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:41.883656 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.870774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.3.82.2:43577)
0925 09:20:41.871765 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.871779 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.872429 (+ 650us) server_negotiation.cc:407] Connection header received
0925 09:20:41.872903 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.872913 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.873206 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.873379 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:41.874517 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.876363 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.880168 (+ 3805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.882106 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.882235 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.882451 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.883172 (+ 721us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.883344 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":660,"threads_started":1}
I20260925 09:20:41.883833 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.870560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43577 (local address 127.0.0.1:42292)
0925 09:20:41.872272 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.872318 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.872586 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.873734 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.873745 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.873788 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:41.874290 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.874306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.876602 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.876611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.879903 (+ 3292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.879925 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.882607 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.882613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.882863 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.882920 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.883071 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.883301 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"mutex_wait_us":1,"thread_start_us":1267,"threads_started":1}
I20260925 09:20:41.888005 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.871579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36624 (local address 127.3.82.3:44991)
0925 09:20:41.876962 (+ 5383us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.876969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.878696 (+ 1727us) server_negotiation.cc:407] Connection header received
0925 09:20:41.878816 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.878829 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.879158 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.879396 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:41.880602 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.882272 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.885384 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.886961 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.887100 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.887337 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.887551 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.887761 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5210,"thread_start_us":5024,"threads_started":1}
I20260925 09:20:41.888046 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.871109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44991 (local address 127.0.0.1:36624)
0925 09:20:41.878370 (+ 7261us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.878395 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.878534 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.879719 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.879726 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.879761 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:41.880386 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.880401 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.882413 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.882421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.884584 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.884599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.887189 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.887196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.887370 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.887404 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.887507 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.887659 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7118,"mutex_wait_us":1,"thread_start_us":6973,"threads_started":1}
I20260925 09:20:41.888012 11282 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7f58ded13a4608b05192ca3d67aaf8 (DEFAULT_TABLE table=test-table [id=fc95b8d9d94941ac9e155742693bdaab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:41.889158 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.872756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:35583 (local address 127.0.0.1:39354)
0925 09:20:41.879878 (+ 7122us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:41.879903 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:41.880123 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:41.880871 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:41.880875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:41.880900 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:41.881277 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.881289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.883083 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.883089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.885230 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:41.885242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.888356 (+ 3114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.888362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:41.888569 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.888601 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:20:41.888692 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 09:20:41.888821 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6931,"thread_start_us":6718,"threads_started":1}
I20260925 09:20:41.889612 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:41.873273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.3.82.1:35583)
0925 09:20:41.879291 (+ 6018us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:41.879307 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:41.880022 (+ 715us) server_negotiation.cc:407] Connection header received
0925 09:20:41.880278 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:41.880294 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:41.880630 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:41.880839 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:41.881498 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.882912 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.885410 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:41.886961 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:41.887102 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:41.887311 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:41.888824 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0925 09:20:41.889008 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5776,"thread_start_us":5600,"threads_started":1}
I20260925 09:20:41.889694 11282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7f58ded13a4608b05192ca3d67aaf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:41.891656 11422 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7f58ded13a4608b05192ca3d67aaf8 (DEFAULT_TABLE table=test-table [id=fc95b8d9d94941ac9e155742693bdaab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:41.892899 11422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7f58ded13a4608b05192ca3d67aaf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:41.893337 11142 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7f58ded13a4608b05192ca3d67aaf8 (DEFAULT_TABLE table=test-table [id=fc95b8d9d94941ac9e155742693bdaab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:41.894845 11142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7f58ded13a4608b05192ca3d67aaf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:41.905457 11516 tablet_bootstrap.cc:492] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Bootstrap starting.
I20260925 09:20:41.907675 11517 tablet_bootstrap.cc:492] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Bootstrap starting.
I20260925 09:20:41.910259 11516 tablet_bootstrap.cc:654] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:41.911091 11517 tablet_bootstrap.cc:654] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:41.911608 11516 log.cc:824] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:41.912354 11517 log.cc:824] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:41.914388 11518 tablet_bootstrap.cc:492] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Bootstrap starting.
I20260925 09:20:41.915282 11516 tablet_bootstrap.cc:492] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: No bootstrap required, opened a new log
I20260925 09:20:41.915589 11516 ts_tablet_manager.cc:1397] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260925 09:20:41.916152 11517 tablet_bootstrap.cc:492] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: No bootstrap required, opened a new log
I20260925 09:20:41.916446 11517 ts_tablet_manager.cc:1397] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 09:20:41.917618 11518 tablet_bootstrap.cc:654] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:41.919021 11518 log.cc:824] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:41.921918 11518 tablet_bootstrap.cc:492] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: No bootstrap required, opened a new log
I20260925 09:20:41.922228 11518 ts_tablet_manager.cc:1397] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 09:20:41.927634 11516 raft_consensus.cc:348] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:41.927636 11517 raft_consensus.cc:348] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:41.928143 11517 raft_consensus.cc:374] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:41.928141 11516 raft_consensus.cc:374] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:41.928251 11516 raft_consensus.cc:729] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Initialized, Role: FOLLOWER
I20260925 09:20:41.928251 11517 raft_consensus.cc:729] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Initialized, Role: FOLLOWER
I20260925 09:20:41.928973 11517 consensus_queue.cc:260] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:41.928974 11516 consensus_queue.cc:260] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:41.930744 11518 raft_consensus.cc:348] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:41.931141 11518 raft_consensus.cc:374] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:41.931262 11518 raft_consensus.cc:729] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4508eaa827384d0cbf0150dda973a447, State: Initialized, Role: FOLLOWER
I20260925 09:20:41.932050 11518 consensus_queue.cc:260] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:41.937688 11517 ts_tablet_manager.cc:1428] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Time spent starting tablet: real 0.021s user 0.013s sys 0.010s
I20260925 09:20:41.938875 11492 heartbeater.cc:503] Master 127.3.82.62:45477 was elected leader, sending a full tablet report...
I20260925 09:20:41.939599 11516 ts_tablet_manager.cc:1428] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent starting tablet: real 0.024s user 0.016s sys 0.007s
I20260925 09:20:41.940141 11518 ts_tablet_manager.cc:1428] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Time spent starting tablet: real 0.018s user 0.010s sys 0.006s
I20260925 09:20:41.940454 11352 heartbeater.cc:503] Master 127.3.82.62:45477 was elected leader, sending a full tablet report...
W20260925 09:20:42.013515 11493 tablet.cc:2369] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:42.102814 11523 raft_consensus.cc:482] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:42.103227 11523 raft_consensus.cc:504] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.104977 11523 leader_election.cc:288] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577), 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583)
I20260925 09:20:42.111794 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.105394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43577 (local address 127.3.82.3:58725)
0925 09:20:42.105762 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.105783 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.106000 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.106541 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.106546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.106582 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:42.106811 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.106826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.108128 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.108132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.109934 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.109949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.111205 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.111210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.111350 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.111375 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.111451 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.111571 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260925 09:20:42.111900 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.105554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:58725 (local address 127.3.82.2:43577)
0925 09:20:42.105785 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.105792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.105923 (+ 131us) server_negotiation.cc:407] Connection header received
0925 09:20:42.106200 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.106208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.106317 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.106438 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:42.107018 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.107944 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.110119 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.110998 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.111195 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.111313 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.111499 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.111634 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 09:20:42.112988 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.106014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:35583 (local address 127.3.82.3:33995)
0925 09:20:42.106756 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.106779 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.106911 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.107297 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.107300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.107324 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:42.107714 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.107724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.108921 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.108927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.110703 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.110714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.112386 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.112393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.112582 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.112607 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.112679 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.112770 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":361,"threads_started":1}
I20260925 09:20:42.113116 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.106181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:33995 (local address 127.3.82.1:35583)
0925 09:20:42.106433 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.106439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.106959 (+ 520us) server_negotiation.cc:407] Connection header received
0925 09:20:42.107041 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.107046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.107140 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.107246 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:42.107896 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.108763 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.110874 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.112177 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.112299 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.112370 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.112735 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.112873 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 09:20:42.113173 11302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7f58ded13a4608b05192ca3d67aaf8" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32836bcc18a34ac784457e2922590f58" is_pre_election: true
I20260925 09:20:42.113749 11302 raft_consensus.cc:2454] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 0.
I20260925 09:20:42.114125 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7f58ded13a4608b05192ca3d67aaf8" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447" is_pre_election: true
I20260925 09:20:42.114555 11372 leader_election.cc:302] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:42.114765 11162 raft_consensus.cc:2454] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 0.
I20260925 09:20:42.114964 11523 raft_consensus.cc:2781] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:42.115129 11523 raft_consensus.cc:482] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:42.115242 11523 raft_consensus.cc:3037] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:20:42.117964 11213 tablet.cc:2369] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:42.117969 11523 raft_consensus.cc:504] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.118721 11523 leader_election.cc:288] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 election: Requested vote from peers 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577), 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583)
I20260925 09:20:42.119356 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7f58ded13a4608b05192ca3d67aaf8" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447"
I20260925 09:20:42.119410 11302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7f58ded13a4608b05192ca3d67aaf8" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32836bcc18a34ac784457e2922590f58"
I20260925 09:20:42.119567 11162 raft_consensus.cc:3037] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.119707 11302 raft_consensus.cc:3037] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.121886 11162 raft_consensus.cc:2454] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 1.
I20260925 09:20:42.122392 11372 leader_election.cc:302] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4508eaa827384d0cbf0150dda973a447, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:42.122745 11523 raft_consensus.cc:2781] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:42.123018 11302 raft_consensus.cc:2454] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 1.
I20260925 09:20:42.123376 11523 raft_consensus.cc:686] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 LEADER]: Becoming Leader. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Running, Role: LEADER
I20260925 09:20:42.124089 11523 consensus_queue.cc:237] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.129815 11004 catalog_manager.cc:5495] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3). New cstate: current_term: 1 leader_uuid: "89d9c87e3619401c9f907fb38e8fee3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } health_report { overall_health: HEALTHY } } }
W20260925 09:20:42.187570 11353 tablet.cc:2369] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:20:42.189234 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.183759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.3.82.3:44991)
0925 09:20:42.183947 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.183954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.183974 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:20:42.184315 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.184319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.184408 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.184570 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:42.184953 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.185649 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.187073 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.187507 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.187640 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.188127 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:42.188669 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:42.188827 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.188884 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.188968 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:20:42.199054 11302 raft_consensus.cc:1261] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Refusing update from remote peer 89d9c87e3619401c9f907fb38e8fee3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:42.199051 11162 raft_consensus.cc:1261] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Refusing update from remote peer 89d9c87e3619401c9f907fb38e8fee3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:42.200083 11526 consensus_queue.cc:1035] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:42.200711 11523 consensus_queue.cc:1035] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:42.212327 11529 mvcc.cc:205] Tried to move back new op lower bound from 7333183660836143104 to 7333183660545232896. Current Snapshot: MvccSnapshot[applied={T|T < 7333183660836143104}]
I20260925 09:20:42.218037 11531 mvcc.cc:205] Tried to move back new op lower bound from 7333183660836143104 to 7333183660545232896. Current Snapshot: MvccSnapshot[applied={T|T < 7333183660836143104 or (T in {7333183660836143104})}]
I20260925 09:20:42.233963 11005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:42.234264 11005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:42.238024 11005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Sending DeleteTablet for 3 replicas of tablet 7c7f58ded13a4608b05192ca3d67aaf8
I20260925 09:20:42.239557 11282 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c7f58ded13a4608b05192ca3d67aaf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:42 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:42292
I20260925 09:20:42.239698 11142 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c7f58ded13a4608b05192ca3d67aaf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:42 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:39354
I20260925 09:20:42.239785 11422 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c7f58ded13a4608b05192ca3d67aaf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:42 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:36624
I20260925 09:20:42.242771 11005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:42.243548 11005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:42.253109 11541 tablet_replica.cc:330] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: stopping tablet replica
I20260925 09:20:42.253737 11541 raft_consensus.cc:2229] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:42.255076 11421 tablet_service.cc:1467] Processing CreateTablet for tablet dcecb178acc047aaa06d5bae97a42176 (DEFAULT_TABLE table=test-table [id=7363aefc873a4921a46e23b50af1ed6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:42.255046 11281 tablet_service.cc:1467] Processing CreateTablet for tablet dcecb178acc047aaa06d5bae97a42176 (DEFAULT_TABLE table=test-table [id=7363aefc873a4921a46e23b50af1ed6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:42.255705 11421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcecb178acc047aaa06d5bae97a42176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:42.255834 11281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcecb178acc047aaa06d5bae97a42176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:42.258411 11141 tablet_service.cc:1467] Processing CreateTablet for tablet dcecb178acc047aaa06d5bae97a42176 (DEFAULT_TABLE table=test-table [id=7363aefc873a4921a46e23b50af1ed6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:42.259057 11141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcecb178acc047aaa06d5bae97a42176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:42.260470 11539 tablet_replica.cc:330] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: stopping tablet replica
I20260925 09:20:42.261152 11539 raft_consensus.cc:2229] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:42.261734 11539 raft_consensus.cc:2258] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:42.264055 11517 tablet_bootstrap.cc:492] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Bootstrap starting.
I20260925 09:20:42.265759 11517 tablet_bootstrap.cc:654] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:42.270437 11539 ts_tablet_manager.cc:1905] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:42.275667 11516 tablet_bootstrap.cc:492] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Bootstrap starting.
I20260925 09:20:42.277952 11516 tablet_bootstrap.cc:654] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:42.283321 11539 ts_tablet_manager.cc:1918] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:42.283460 11518 tablet_bootstrap.cc:492] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Bootstrap starting.
I20260925 09:20:42.283528 11539 log.cc:1196] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/7c7f58ded13a4608b05192ca3d67aaf8
I20260925 09:20:42.284107 11539 ts_tablet_manager.cc:1939] T 7c7f58ded13a4608b05192ca3d67aaf8 P 32836bcc18a34ac784457e2922590f58: Deleting consensus metadata
I20260925 09:20:42.285843 11518 tablet_bootstrap.cc:654] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:42.286080 10989 catalog_manager.cc:4841] TS 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577): tablet 7c7f58ded13a4608b05192ca3d67aaf8 (table test-table [id=fc95b8d9d94941ac9e155742693bdaab]) successfully deleted
I20260925 09:20:42.288817 11517 tablet_bootstrap.cc:492] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: No bootstrap required, opened a new log
I20260925 09:20:42.288995 11517 ts_tablet_manager.cc:1397] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.005s
I20260925 09:20:42.289943 11517 raft_consensus.cc:348] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.290236 11517 raft_consensus.cc:374] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:42.290328 11517 raft_consensus.cc:729] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Initialized, Role: FOLLOWER
I20260925 09:20:42.290647 11517 consensus_queue.cc:260] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.293705 11517 ts_tablet_manager.cc:1428] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 09:20:42.296627 11540 tablet_replica.cc:330] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: stopping tablet replica
I20260925 09:20:42.296674 11541 raft_consensus.cc:2258] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:42.297176 11540 raft_consensus.cc:2229] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:42.297492 11518 tablet_bootstrap.cc:492] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: No bootstrap required, opened a new log
I20260925 09:20:42.297706 11518 ts_tablet_manager.cc:1397] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.005s
I20260925 09:20:42.297780 11540 raft_consensus.cc:2258] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:42.299073 11541 ts_tablet_manager.cc:1905] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:42.299726 11518 raft_consensus.cc:348] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.299990 11518 raft_consensus.cc:374] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:42.300138 11518 raft_consensus.cc:729] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4508eaa827384d0cbf0150dda973a447, State: Initialized, Role: FOLLOWER
I20260925 09:20:42.300465 11518 consensus_queue.cc:260] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.299126 11516 tablet_bootstrap.cc:492] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: No bootstrap required, opened a new log
I20260925 09:20:42.300945 11516 ts_tablet_manager.cc:1397] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.005s
I20260925 09:20:42.301047 11518 ts_tablet_manager.cc:1428] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260925 09:20:42.301805 11516 raft_consensus.cc:348] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.302048 11516 raft_consensus.cc:374] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:42.302155 11516 raft_consensus.cc:729] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Initialized, Role: FOLLOWER
I20260925 09:20:42.302484 11516 consensus_queue.cc:260] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.302642 11540 ts_tablet_manager.cc:1905] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:42.306017 11516 ts_tablet_manager.cc:1428] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260925 09:20:42.309445 11540 ts_tablet_manager.cc:1918] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:42.309643 11540 log.cc:1196] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/7c7f58ded13a4608b05192ca3d67aaf8
I20260925 09:20:42.310253 11540 ts_tablet_manager.cc:1939] T 7c7f58ded13a4608b05192ca3d67aaf8 P 4508eaa827384d0cbf0150dda973a447: Deleting consensus metadata
I20260925 09:20:42.311895 11541 ts_tablet_manager.cc:1918] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:42.311930 10989 catalog_manager.cc:4841] TS 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583): tablet 7c7f58ded13a4608b05192ca3d67aaf8 (table test-table [id=fc95b8d9d94941ac9e155742693bdaab]) successfully deleted
I20260925 09:20:42.312089 11541 log.cc:1196] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/7c7f58ded13a4608b05192ca3d67aaf8
I20260925 09:20:42.312642 11541 ts_tablet_manager.cc:1939] T 7c7f58ded13a4608b05192ca3d67aaf8 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting consensus metadata
I20260925 09:20:42.314360 10991 catalog_manager.cc:4841] TS 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991): tablet 7c7f58ded13a4608b05192ca3d67aaf8 (table test-table [id=fc95b8d9d94941ac9e155742693bdaab]) successfully deleted
I20260925 09:20:42.454818 11524 raft_consensus.cc:482] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:42.455063 11524 raft_consensus.cc:504] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.456760 11524 leader_election.cc:288] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577), 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:42.466051 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.457354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48795 (local address 127.3.82.2:43577)
0925 09:20:42.457529 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.457534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.458537 (+ 1003us) server_negotiation.cc:407] Connection header received
0925 09:20:42.458745 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.458753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.458851 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.458916 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:42.459611 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.460351 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.463552 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.464927 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.465087 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.465186 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.465567 (+ 381us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.465731 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:20:42.466167 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.457113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43577 (local address 127.3.82.1:48795)
0925 09:20:42.458436 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.458455 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.458576 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.459130 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.459134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.459165 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:42.459454 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.459462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.460483 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.460492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.463343 (+ 2851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.463363 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.465187 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.465193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.465351 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.465380 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.465529 (+ 149us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.465689 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1186,"mutex_wait_us":680,"thread_start_us":538,"threads_started":1}
I20260925 09:20:42.467864 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.457163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:33887 (local address 127.3.82.3:44991)
0925 09:20:42.457356 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.457365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.458100 (+ 735us) server_negotiation.cc:407] Connection header received
0925 09:20:42.458462 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.458468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.458556 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.458654 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:42.459301 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.460400 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.465301 (+ 4901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.466671 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.466823 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.466936 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.467409 (+ 473us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.467555 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:20:42.467741 11302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcecb178acc047aaa06d5bae97a42176" candidate_uuid: "4508eaa827384d0cbf0150dda973a447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32836bcc18a34ac784457e2922590f58" is_pre_election: true
I20260925 09:20:42.468078 11302 raft_consensus.cc:2454] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4508eaa827384d0cbf0150dda973a447 in term 0.
I20260925 09:20:42.468263 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.457105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44991 (local address 127.3.82.1:33887)
0925 09:20:42.457976 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.458002 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.458240 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.458685 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.458693 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.458719 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:42.459114 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.459127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.460551 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.460555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.463341 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.463363 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.466962 (+ 3599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.466969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.467113 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.467142 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.467350 (+ 208us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.467589 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":137,"thread_start_us":538,"threads_started":1}
I20260925 09:20:42.468956 11092 leader_election.cc:302] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 4508eaa827384d0cbf0150dda973a447; no voters:
I20260925 09:20:42.469452 11524 raft_consensus.cc:2781] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:42.469236 11447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcecb178acc047aaa06d5bae97a42176" candidate_uuid: "4508eaa827384d0cbf0150dda973a447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d9c87e3619401c9f907fb38e8fee3a" is_pre_election: true
I20260925 09:20:42.469597 11524 raft_consensus.cc:482] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:42.469713 11524 raft_consensus.cc:3037] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.469817 11447 raft_consensus.cc:2454] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4508eaa827384d0cbf0150dda973a447 in term 0.
I20260925 09:20:42.472600 11524 raft_consensus.cc:504] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.473354 11524 leader_election.cc:288] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [CANDIDATE]: Term 1 election: Requested vote from peers 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577), 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:42.474044 11302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcecb178acc047aaa06d5bae97a42176" candidate_uuid: "4508eaa827384d0cbf0150dda973a447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32836bcc18a34ac784457e2922590f58"
I20260925 09:20:42.474059 11447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcecb178acc047aaa06d5bae97a42176" candidate_uuid: "4508eaa827384d0cbf0150dda973a447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d9c87e3619401c9f907fb38e8fee3a"
I20260925 09:20:42.474304 11302 raft_consensus.cc:3037] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.474310 11447 raft_consensus.cc:3037] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.477986 11302 raft_consensus.cc:2454] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4508eaa827384d0cbf0150dda973a447 in term 1.
I20260925 09:20:42.478229 11447 raft_consensus.cc:2454] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4508eaa827384d0cbf0150dda973a447 in term 1.
I20260925 09:20:42.478730 11092 leader_election.cc:302] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 4508eaa827384d0cbf0150dda973a447; no voters:
I20260925 09:20:42.479272 11524 raft_consensus.cc:2781] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:42.479835 11524 raft_consensus.cc:686] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 1 LEADER]: Becoming Leader. State: Replica: 4508eaa827384d0cbf0150dda973a447, State: Running, Role: LEADER
I20260925 09:20:42.480598 11524 consensus_queue.cc:237] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.484651 11004 catalog_manager.cc:5495] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4508eaa827384d0cbf0150dda973a447 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "4508eaa827384d0cbf0150dda973a447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } health_report { overall_health: UNKNOWN } } }
Sep 25 09:20:42 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328039, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.62@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:42.538578 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.520495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45477 (local address 127.0.0.1:50998)
0925 09:20:42.521491 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.521508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.521659 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.522124 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.522131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.522857 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:42.523078 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.523087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.524679 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.524687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.525798 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.525811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.526951 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.526956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.527119 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.528072 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:42.528092 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:42.530925 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:42.535404 (+ 4479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:42.535420 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:42.535427 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:42.536028 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:42.536586 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:42.536589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:42.536592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:42.536785 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:42.537519 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:42.537527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:42.537758 (+ 231us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.538012 (+ 254us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.538255 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":659,"threads_started":1}
I20260925 09:20:42.538956 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.520509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.3.82.62:45477)
0925 09:20:42.521230 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.521237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.521608 (+ 371us) server_negotiation.cc:407] Connection header received
0925 09:20:42.521790 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.521796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.521909 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.522046 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:42.523242 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.524463 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.525995 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.526672 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.526821 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.531236 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:42.531245 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:42.531250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:42.531293 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:42.535179 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:42.536193 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:42.536199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:42.536203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:42.536377 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:42.536912 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:42.536920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:42.536923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:42.537361 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:42.537535 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.538157 (+ 622us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.538419 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":531,"thread_start_us":357,"threads_started":1}
I20260925 09:20:42.545059 11004 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:50998:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 09:20:42.546603 11004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:42.554699 11142 tablet_service.cc:1467] Processing CreateTablet for tablet 74b0411a795d47889f56bd89c7b37ad8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a971630b5594bd0955b4f02fb6db375]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:20:42.555171 11281 tablet_service.cc:1467] Processing CreateTablet for tablet 74b0411a795d47889f56bd89c7b37ad8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a971630b5594bd0955b4f02fb6db375]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:20:42.555387 11142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b0411a795d47889f56bd89c7b37ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:42.555668 11281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b0411a795d47889f56bd89c7b37ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:42.562376 11421 tablet_service.cc:1467] Processing CreateTablet for tablet 74b0411a795d47889f56bd89c7b37ad8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a971630b5594bd0955b4f02fb6db375]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 09:20:42.563014 11421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b0411a795d47889f56bd89c7b37ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:42.563890 11518 tablet_bootstrap.cc:492] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: Bootstrap starting.
I20260925 09:20:42.565858 11518 tablet_bootstrap.cc:654] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:42.570781 11517 tablet_bootstrap.cc:492] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: Bootstrap starting.
I20260925 09:20:42.572633 11517 tablet_bootstrap.cc:654] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:42.573123 11518 tablet_bootstrap.cc:492] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: No bootstrap required, opened a new log
I20260925 09:20:42.573359 11518 ts_tablet_manager.cc:1397] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260925 09:20:42.574532 11518 raft_consensus.cc:348] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.575037 11516 tablet_bootstrap.cc:492] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: Bootstrap starting.
I20260925 09:20:42.574913 11518 raft_consensus.cc:374] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:42.575244 11518 raft_consensus.cc:729] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4508eaa827384d0cbf0150dda973a447, State: Initialized, Role: FOLLOWER
I20260925 09:20:42.576345 11518 consensus_queue.cc:260] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.576829 11524 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.577075 11516 tablet_bootstrap.cc:654] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:42.577167 11524 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.578261 11517 tablet_bootstrap.cc:492] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: No bootstrap required, opened a new log
I20260925 09:20:42.578447 11518 ts_tablet_manager.cc:1428] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:20:42.578517 11517 ts_tablet_manager.cc:1397] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20260925 09:20:42.579416 11517 raft_consensus.cc:348] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.579670 11517 raft_consensus.cc:374] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:42.579746 11517 raft_consensus.cc:729] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Initialized, Role: FOLLOWER
I20260925 09:20:42.581655 11517 consensus_queue.cc:260] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.582350 11517 ts_tablet_manager.cc:1428] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 09:20:42.582425 11522 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.582867 11522 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.587580 11516 tablet_bootstrap.cc:492] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: No bootstrap required, opened a new log
I20260925 09:20:42.587798 11516 ts_tablet_manager.cc:1397] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260925 09:20:42.588688 11516 raft_consensus.cc:348] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.588967 11516 raft_consensus.cc:374] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:42.589063 11516 raft_consensus.cc:729] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Initialized, Role: FOLLOWER
I20260925 09:20:42.589373 11516 consensus_queue.cc:260] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.590108 11516 ts_tablet_manager.cc:1428] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260925 09:20:42.589946 11537 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.590387 11537 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.863781 11522 raft_consensus.cc:482] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:42.864024 11522 raft_consensus.cc:504] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.865672 11522 leader_election.cc:288] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583), 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:42.870251 11524 consensus_queue.cc:1035] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:42.874717 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.865811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:35583 (local address 127.3.82.2:60791)
0925 09:20:42.867232 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.867255 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.867418 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.867849 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.867854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.867892 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:42.868566 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.868580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.870011 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.870016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.871953 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.871970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.873638 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.873647 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.873969 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.874020 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.874118 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.874272 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"mutex_wait_us":221,"spinlock_wait_cycles":896,"thread_start_us":1285,"threads_started":1}
I20260925 09:20:42.874749 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.866480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:54943 (local address 127.3.82.3:44991)
0925 09:20:42.866680 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.866688 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.867466 (+ 778us) server_negotiation.cc:407] Connection header received
0925 09:20:42.867624 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.867630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.867717 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.867834 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:42.868585 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.869384 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.872095 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.873401 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.873534 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.873649 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.874282 (+ 633us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.874439 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:20:42.875375 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.866493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44991 (local address 127.3.82.2:54943)
0925 09:20:42.867371 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.867385 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.867482 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.867849 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.867854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.867899 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:20:42.868333 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.868346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.869516 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.869523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.871926 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.871946 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.873814 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.873863 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.874082 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.874114 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.874963 (+ 849us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.875106 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":119,"spinlock_wait_cycles":4864,"thread_start_us":467,"threads_started":1}
I20260925 09:20:42.875365 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.865926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:60791 (local address 127.3.82.1:35583)
0925 09:20:42.867137 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.867145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.867346 (+ 201us) server_negotiation.cc:407] Connection header received
0925 09:20:42.867624 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.867630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.867710 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.867832 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:20:42.868720 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.869801 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.872171 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.873401 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.873534 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.873643 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.874947 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.875129 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1042,"thread_start_us":772,"threads_started":1}
I20260925 09:20:42.875952 11447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b0411a795d47889f56bd89c7b37ad8" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d9c87e3619401c9f907fb38e8fee3a" is_pre_election: true
I20260925 09:20:42.876194 11447 raft_consensus.cc:2454] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 0.
I20260925 09:20:42.876507 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b0411a795d47889f56bd89c7b37ad8" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447" is_pre_election: true
I20260925 09:20:42.876756 11162 raft_consensus.cc:2454] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 0.
I20260925 09:20:42.876693 11234 leader_election.cc:302] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:42.877182 11522 raft_consensus.cc:2781] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:42.877324 11522 raft_consensus.cc:482] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:42.877442 11522 raft_consensus.cc:3037] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.880246 11547 consensus_queue.cc:1035] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:42.880717 11522 raft_consensus.cc:504] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.881870 11522 leader_election.cc:288] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 election: Requested vote from peers 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583), 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:42.882053 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b0411a795d47889f56bd89c7b37ad8" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447"
I20260925 09:20:42.882212 11447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b0411a795d47889f56bd89c7b37ad8" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d9c87e3619401c9f907fb38e8fee3a"
I20260925 09:20:42.882300 11162 raft_consensus.cc:3037] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.882421 11447 raft_consensus.cc:3037] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:42.884696 11447 raft_consensus.cc:2454] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 1.
I20260925 09:20:42.885314 11234 leader_election.cc:302] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:42.885650 11565 raft_consensus.cc:2781] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:42.886179 11565 raft_consensus.cc:686] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [term 1 LEADER]: Becoming Leader. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Running, Role: LEADER
I20260925 09:20:42.886632 11162 raft_consensus.cc:2454] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 1.
I20260925 09:20:42.886852 11565 consensus_queue.cc:237] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:42.888985 11522 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: TxnStatusTablet state changed. Reason: New leader 32836bcc18a34ac784457e2922590f58. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.889314 11522 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:20:42.890005 11569 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 09:20:42.890278 11569 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 09:20:42.893903 11004 catalog_manager.cc:5495] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32836bcc18a34ac784457e2922590f58 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } health_report { overall_health: UNKNOWN } } }
I20260925 09:20:42.951411 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.943099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43577 (local address 127.0.0.1:42296)
0925 09:20:42.943380 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:42.943402 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:42.943568 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:42.944151 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:42.944158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:42.944174 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:20:42.944424 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.944436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.946110 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.946116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.947895 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:42.947912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.949032 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.949039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:42.949268 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.949386 (+ 118us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:20:42.950650 (+ 1264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:20:42.950714 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 09:20:42.950831 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 09:20:42.951024 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 09:20:42.952214 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.943053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.3.82.2:43577)
0925 09:20:42.943553 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.943560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.943585 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:20:42.943829 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.943836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.943951 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.944115 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:42.944583 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.945966 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.948063 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.948841 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.948990 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.949628 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:42.950522 (+ 894us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:42.950665 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.951784 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.951893 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
I20260925 09:20:42.957073 11447 raft_consensus.cc:1261] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Refusing update from remote peer 32836bcc18a34ac784457e2922590f58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:42.957302 11162 raft_consensus.cc:1261] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Refusing update from remote peer 32836bcc18a34ac784457e2922590f58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:42.957967 11565 consensus_queue.cc:1035] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:42.958377 11522 consensus_queue.cc:1035] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:42.962664 11547 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: TxnStatusTablet state changed. Reason: New leader 32836bcc18a34ac784457e2922590f58. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.963084 11547 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.965766 11567 mvcc.cc:205] Tried to move back new op lower bound from 7333183663943929856 to 7333183663668850688. Current Snapshot: MvccSnapshot[applied={T|T < 7333183663943929856}]
I20260925 09:20:42.968724 11565 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.969168 11565 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:20:42.973606 11537 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: TxnStatusTablet state changed. Reason: New leader 32836bcc18a34ac784457e2922590f58. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.973981 11537 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.974113 11547 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.974373 11547 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 4508eaa827384d0cbf0150dda973a447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.975374 11522 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.975683 11522 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 32836bcc18a34ac784457e2922590f58: This TxnStatusTablet replica's current role is: LEADER
I20260925 09:20:42.979723 11537 tablet_replica.cc:439] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } } }
I20260925 09:20:42.980026 11537 tablet_replica.cc:442] T 74b0411a795d47889f56bd89c7b37ad8 P 89d9c87e3619401c9f907fb38e8fee3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 09:20:42.992112 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:42.985339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.3.82.1:35583)
0925 09:20:42.985568 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:42.985575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:42.985820 (+ 245us) server_negotiation.cc:407] Connection header received
0925 09:20:42.986087 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:42.986091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:42.986189 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:42.986334 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:42.986819 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.987509 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.989692 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:42.990203 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:42.990309 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:42.990832 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:42.991441 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:42.991640 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:42.991757 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 09:20:42.991831 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:20:43.009179 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:43.003416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:54967 (local address 127.3.82.2:43577)
0925 09:20:43.003650 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:43.003656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:43.004229 (+ 573us) server_negotiation.cc:407] Connection header received
0925 09:20:43.004468 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:43.004474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:43.004569 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:43.004670 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:43.005124 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.005988 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.007342 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.007834 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.007978 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.008389 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:43.008509 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:43.008609 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:43.008758 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 09:20:43.008849 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:20:43.009248 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:43.003455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43577 (local address 127.3.82.1:54967)
0925 09:20:43.004080 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:43.004103 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:43.004326 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:43.004710 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:43.004715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:43.004727 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:20:43.004983 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:43.004991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.006112 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.006118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:43.007186 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:43.007198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.007998 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.008004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:43.008146 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.008207 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:20:43.008608 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:20:43.008622 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 09:20:43.008719 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 09:20:43.008867 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":318,"threads_started":1}
I20260925 09:20:43.054167 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:43.046062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:36807 (local address 127.3.82.1:35583)
0925 09:20:43.046273 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:43.046281 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:43.047082 (+ 801us) server_negotiation.cc:407] Connection header received
0925 09:20:43.047346 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:43.047353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:43.047485 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:43.047590 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:43.048631 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.049464 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.051853 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.052526 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.052682 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.053267 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:43.053425 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:43.053526 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:43.053700 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 09:20:43.053792 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:20:43.054384 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:43.046116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:35583 (local address 127.3.82.2:36807)
0925 09:20:43.046965 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:43.046984 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:43.047171 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:43.047762 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:43.047767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:43.047781 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:20:43.048262 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:43.048275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.049682 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.049688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:43.051654 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:43.051673 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.052733 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.052743 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:43.052911 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.053045 (+ 134us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:20:43.053540 (+ 495us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:20:43.053554 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 09:20:43.053667 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 09:20:43.053872 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":447,"threads_started":1}
I20260925 09:20:43.101742 11005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:43.102077 11005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:43.106017 11005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Sending DeleteTablet for 3 replicas of tablet dcecb178acc047aaa06d5bae97a42176
I20260925 09:20:43.107252 11142 tablet_service.cc:1514] Processing DeleteTablet for tablet dcecb178acc047aaa06d5bae97a42176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:43 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:39354
I20260925 09:20:43.107273 11421 tablet_service.cc:1514] Processing DeleteTablet for tablet dcecb178acc047aaa06d5bae97a42176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:43 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:36624
I20260925 09:20:43.107833 11281 tablet_service.cc:1514] Processing DeleteTablet for tablet dcecb178acc047aaa06d5bae97a42176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:43 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:42292
I20260925 09:20:43.120106 11589 tablet_replica.cc:330] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: stopping tablet replica
I20260925 09:20:43.120395 11589 raft_consensus.cc:2229] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:43.120684 11589 raft_consensus.cc:2258] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:43.122139 11589 ts_tablet_manager.cc:1905] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:43.130059 11589 ts_tablet_manager.cc:1918] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:20:43.130242 11589 log.cc:1196] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/dcecb178acc047aaa06d5bae97a42176
I20260925 09:20:43.130704 11589 ts_tablet_manager.cc:1939] T dcecb178acc047aaa06d5bae97a42176 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting consensus metadata
I20260925 09:20:43.132038 10991 catalog_manager.cc:4841] TS 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991): tablet dcecb178acc047aaa06d5bae97a42176 (table test-table [id=7363aefc873a4921a46e23b50af1ed6a]) successfully deleted
I20260925 09:20:43.144375 11590 tablet_replica.cc:330] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: stopping tablet replica
I20260925 09:20:43.144649 11590 raft_consensus.cc:2229] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:43.144984 11590 raft_consensus.cc:2258] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:43.145660 11588 tablet_replica.cc:330] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: stopping tablet replica
I20260925 09:20:43.145905 11588 raft_consensus.cc:2229] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:43.146620 11588 raft_consensus.cc:2258] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:43.147854 11588 ts_tablet_manager.cc:1905] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:43.150532 11590 ts_tablet_manager.cc:1905] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:43.155447 11588 ts_tablet_manager.cc:1918] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:20:43.155613 11588 log.cc:1196] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/dcecb178acc047aaa06d5bae97a42176
I20260925 09:20:43.156095 11588 ts_tablet_manager.cc:1939] T dcecb178acc047aaa06d5bae97a42176 P 4508eaa827384d0cbf0150dda973a447: Deleting consensus metadata
I20260925 09:20:43.157610 10989 catalog_manager.cc:4841] TS 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583): tablet dcecb178acc047aaa06d5bae97a42176 (table test-table [id=7363aefc873a4921a46e23b50af1ed6a]) successfully deleted
I20260925 09:20:43.165526 11590 ts_tablet_manager.cc:1918] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:20:43.165707 11590 log.cc:1196] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/dcecb178acc047aaa06d5bae97a42176
I20260925 09:20:43.166258 11590 ts_tablet_manager.cc:1939] T dcecb178acc047aaa06d5bae97a42176 P 32836bcc18a34ac784457e2922590f58: Deleting consensus metadata
I20260925 09:20:43.167688 10989 catalog_manager.cc:4841] TS 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577): tablet dcecb178acc047aaa06d5bae97a42176 (table test-table [id=7363aefc873a4921a46e23b50af1ed6a]) successfully deleted
Sep 25 09:20:43 dist-test-slave-4wm5 krb5kdc[10960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 09:20:43 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:43 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:20:43.279824 11597 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:43.251402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.3.82.62:45477)
0925 09:20:43.264896 (+ 13494us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:43.264901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:43.264918 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:43.264974 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:43.264977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:43.265041 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:43.265187 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:43.267182 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.268975 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.271262 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.271976 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.272127 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.279457 (+ 7330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51002: BlockingRecv error: recv got EOF from 127.0.0.1:51002 (error 108)
Metrics: {"server-negotiator.queue_time_us":13373,"thread_start_us":13233,"threads_started":1}
Sep 25 09:20:43 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM
I20260925 09:20:43.309446 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:43.284408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.3.82.62:45477)
0925 09:20:43.285877 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:43.285887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:43.286923 (+ 1036us) server_negotiation.cc:407] Connection header received
0925 09:20:43.287288 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:43.287296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:43.287400 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:43.287490 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:43.289554 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.290942 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.293292 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.294122 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.294256 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.301698 (+ 7442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:43.301708 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:43.301715 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:43.301811 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:43.306191 (+ 4380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:43.307114 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:43.307120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:43.307124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:43.307275 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:43.307758 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:43.307765 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:43.307768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:43.308079 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:43.308405 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:43.308792 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:20:43.309034 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1258}
I20260925 09:20:43.313236 11005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:43.313946 11005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:43.321184 11421 tablet_service.cc:1467] Processing CreateTablet for tablet fe26b6df4b6942b9a3fe3b330bbb44c3 (DEFAULT_TABLE table=test-table [id=7cfa1fa4883d46de94c17ff9d476c44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:43.321705 11421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe26b6df4b6942b9a3fe3b330bbb44c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:43.326274 11142 tablet_service.cc:1467] Processing CreateTablet for tablet fe26b6df4b6942b9a3fe3b330bbb44c3 (DEFAULT_TABLE table=test-table [id=7cfa1fa4883d46de94c17ff9d476c44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:43.326730 11142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe26b6df4b6942b9a3fe3b330bbb44c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:43.330350 11281 tablet_service.cc:1467] Processing CreateTablet for tablet fe26b6df4b6942b9a3fe3b330bbb44c3 (DEFAULT_TABLE table=test-table [id=7cfa1fa4883d46de94c17ff9d476c44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:43.330859 11281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe26b6df4b6942b9a3fe3b330bbb44c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:43.342638 11604 tablet_bootstrap.cc:492] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Bootstrap starting.
I20260925 09:20:43.345340 11604 tablet_bootstrap.cc:654] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:43.349045 11604 tablet_bootstrap.cc:492] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: No bootstrap required, opened a new log
I20260925 09:20:43.349249 11604 ts_tablet_manager.cc:1397] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 09:20:43.350373 11605 tablet_bootstrap.cc:492] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Bootstrap starting.
I20260925 09:20:43.350215 11604 raft_consensus.cc:348] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.350508 11604 raft_consensus.cc:374] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:43.350633 11604 raft_consensus.cc:729] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4508eaa827384d0cbf0150dda973a447, State: Initialized, Role: FOLLOWER
I20260925 09:20:43.350944 11604 consensus_queue.cc:260] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.352319 11605 tablet_bootstrap.cc:654] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:43.356747 11604 ts_tablet_manager.cc:1428] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260925 09:20:43.357662 11606 tablet_bootstrap.cc:492] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Bootstrap starting.
I20260925 09:20:43.360764 11606 tablet_bootstrap.cc:654] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:43.363830 11605 tablet_bootstrap.cc:492] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: No bootstrap required, opened a new log
I20260925 09:20:43.364100 11605 ts_tablet_manager.cc:1397] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260925 09:20:43.365023 11605 raft_consensus.cc:348] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.365423 11605 raft_consensus.cc:374] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:43.365594 11605 raft_consensus.cc:729] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Initialized, Role: FOLLOWER
I20260925 09:20:43.365972 11605 consensus_queue.cc:260] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.368521 11605 ts_tablet_manager.cc:1428] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 09:20:43.375917 11606 tablet_bootstrap.cc:492] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: No bootstrap required, opened a new log
I20260925 09:20:43.376116 11606 ts_tablet_manager.cc:1397] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.007s
I20260925 09:20:43.377041 11606 raft_consensus.cc:348] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.377337 11606 raft_consensus.cc:374] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:43.377451 11606 raft_consensus.cc:729] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Initialized, Role: FOLLOWER
I20260925 09:20:43.377781 11606 consensus_queue.cc:260] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.378511 11606 ts_tablet_manager.cc:1428] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 09:20:43.499943 11565 raft_consensus.cc:482] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:43.500176 11565 raft_consensus.cc:504] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.501036 11565 leader_election.cc:288] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583), 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:43.501634 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe26b6df4b6942b9a3fe3b330bbb44c3" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447" is_pre_election: true
I20260925 09:20:43.501683 11446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe26b6df4b6942b9a3fe3b330bbb44c3" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d9c87e3619401c9f907fb38e8fee3a" is_pre_election: true
I20260925 09:20:43.501972 11162 raft_consensus.cc:2454] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 0.
I20260925 09:20:43.501955 11446 raft_consensus.cc:2454] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 0.
I20260925 09:20:43.502540 11234 leader_election.cc:302] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 4508eaa827384d0cbf0150dda973a447, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:43.502931 11565 raft_consensus.cc:2781] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:43.503091 11565 raft_consensus.cc:482] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:43.503206 11565 raft_consensus.cc:3037] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:43.506718 11565 raft_consensus.cc:504] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.507647 11565 leader_election.cc:288] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 election: Requested vote from peers 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583), 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991)
I20260925 09:20:43.508320 11446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe26b6df4b6942b9a3fe3b330bbb44c3" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d9c87e3619401c9f907fb38e8fee3a"
I20260925 09:20:43.508344 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe26b6df4b6942b9a3fe3b330bbb44c3" candidate_uuid: "32836bcc18a34ac784457e2922590f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447"
I20260925 09:20:43.508549 11446 raft_consensus.cc:3037] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:43.508607 11162 raft_consensus.cc:3037] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:43.511031 11446 raft_consensus.cc:2454] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 1.
I20260925 09:20:43.511632 11234 leader_election.cc:302] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32836bcc18a34ac784457e2922590f58, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:43.511945 11565 raft_consensus.cc:2781] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:43.512290 11162 raft_consensus.cc:2454] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32836bcc18a34ac784457e2922590f58 in term 1.
I20260925 09:20:43.512156 11565 raft_consensus.cc:686] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 1 LEADER]: Becoming Leader. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Running, Role: LEADER
I20260925 09:20:43.512759 11565 consensus_queue.cc:237] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.515820 11005 catalog_manager.cc:5495] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32836bcc18a34ac784457e2922590f58 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "32836bcc18a34ac784457e2922590f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } health_report { overall_health: UNKNOWN } } }
I20260925 09:20:43.552393 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:43.543379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.3.82.2:43577)
0925 09:20:43.546327 (+ 2948us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:43.546335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:43.546362 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:20:43.546446 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:43.546451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:43.546550 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:43.546675 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:43.547141 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.548322 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.549905 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:43.550660 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:43.550825 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:43.551358 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:43.551544 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:43.551707 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:43.551908 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 09:20:43.552006 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2782,"thread_start_us":2637,"threads_started":1}
I20260925 09:20:43.556481 11162 raft_consensus.cc:1261] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Refusing update from remote peer 32836bcc18a34ac784457e2922590f58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:43.556463 11446 raft_consensus.cc:1261] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Refusing update from remote peer 32836bcc18a34ac784457e2922590f58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:43.557255 11565 consensus_queue.cc:1035] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:43.557588 11522 consensus_queue.cc:1035] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:43.579487 11005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:43.579777 11005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:43.583877 11005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Sending DeleteTablet for 3 replicas of tablet fe26b6df4b6942b9a3fe3b330bbb44c3
I20260925 09:20:43.585043 11281 tablet_service.cc:1514] Processing DeleteTablet for tablet fe26b6df4b6942b9a3fe3b330bbb44c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:43 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:42292
I20260925 09:20:43.585042 11142 tablet_service.cc:1514] Processing DeleteTablet for tablet fe26b6df4b6942b9a3fe3b330bbb44c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:43 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:39354
I20260925 09:20:43.585343 11590 tablet_replica.cc:330] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: stopping tablet replica
I20260925 09:20:43.585400 11588 tablet_replica.cc:330] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: stopping tablet replica
I20260925 09:20:43.585523 11590 raft_consensus.cc:2229] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:43.585628 11588 raft_consensus.cc:2229] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:43.585928 11588 raft_consensus.cc:2258] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:43.586071 11590 raft_consensus.cc:2258] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:43.587319 11590 ts_tablet_manager.cc:1905] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:43.587598 11588 ts_tablet_manager.cc:1905] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:43.594873 11590 ts_tablet_manager.cc:1918] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:43.595075 11590 log.cc:1196] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/fe26b6df4b6942b9a3fe3b330bbb44c3
I20260925 09:20:43.596200 11421 tablet_service.cc:1514] Processing DeleteTablet for tablet fe26b6df4b6942b9a3fe3b330bbb44c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:43 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:36624
I20260925 09:20:43.596241 11588 ts_tablet_manager.cc:1918] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:43.596580 11589 tablet_replica.cc:330] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: stopping tablet replica
I20260925 09:20:43.596694 11588 log.cc:1196] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/fe26b6df4b6942b9a3fe3b330bbb44c3
I20260925 09:20:43.596804 11589 raft_consensus.cc:2229] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:43.597088 11589 raft_consensus.cc:2258] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:43.597396 11588 ts_tablet_manager.cc:1939] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 4508eaa827384d0cbf0150dda973a447: Deleting consensus metadata
I20260925 09:20:43.598399 11589 ts_tablet_manager.cc:1905] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:43.598935 10989 catalog_manager.cc:4841] TS 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583): tablet fe26b6df4b6942b9a3fe3b330bbb44c3 (table test-table [id=7cfa1fa4883d46de94c17ff9d476c44c]) successfully deleted
I20260925 09:20:43.603047 11590 ts_tablet_manager.cc:1939] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 32836bcc18a34ac784457e2922590f58: Deleting consensus metadata
I20260925 09:20:43.604557 10989 catalog_manager.cc:4841] TS 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577): tablet fe26b6df4b6942b9a3fe3b330bbb44c3 (table test-table [id=7cfa1fa4883d46de94c17ff9d476c44c]) successfully deleted
I20260925 09:20:43.605325 11005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:20:43.606369 11005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:20:43.607820 11589 ts_tablet_manager.cc:1918] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:20:43.608017 11589 log.cc:1196] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/fe26b6df4b6942b9a3fe3b330bbb44c3
I20260925 09:20:43.608461 11589 ts_tablet_manager.cc:1939] T fe26b6df4b6942b9a3fe3b330bbb44c3 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting consensus metadata
I20260925 09:20:43.609724 10991 catalog_manager.cc:4841] TS 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991): tablet fe26b6df4b6942b9a3fe3b330bbb44c3 (table test-table [id=7cfa1fa4883d46de94c17ff9d476c44c]) successfully deleted
I20260925 09:20:43.616955 11421 tablet_service.cc:1467] Processing CreateTablet for tablet 3d908d6b9c0a438d8d3b910eaea4b756 (DEFAULT_TABLE table=test-table [id=bc3f338fc4b04f36bb22f4e5847c2afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:43.617484 11421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d908d6b9c0a438d8d3b910eaea4b756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:43.625487 11142 tablet_service.cc:1467] Processing CreateTablet for tablet 3d908d6b9c0a438d8d3b910eaea4b756 (DEFAULT_TABLE table=test-table [id=bc3f338fc4b04f36bb22f4e5847c2afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:43.626132 11142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d908d6b9c0a438d8d3b910eaea4b756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:43.630389 11281 tablet_service.cc:1467] Processing CreateTablet for tablet 3d908d6b9c0a438d8d3b910eaea4b756 (DEFAULT_TABLE table=test-table [id=bc3f338fc4b04f36bb22f4e5847c2afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:20:43.631019 11281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d908d6b9c0a438d8d3b910eaea4b756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:43.641191 11605 tablet_bootstrap.cc:492] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Bootstrap starting.
I20260925 09:20:43.643225 11605 tablet_bootstrap.cc:654] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:43.643325 11606 tablet_bootstrap.cc:492] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Bootstrap starting.
I20260925 09:20:43.645375 11606 tablet_bootstrap.cc:654] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:43.645705 11605 tablet_bootstrap.cc:492] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: No bootstrap required, opened a new log
I20260925 09:20:43.645936 11605 ts_tablet_manager.cc:1397] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:20:43.646858 11605 raft_consensus.cc:348] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.647251 11605 raft_consensus.cc:374] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:43.647341 11605 raft_consensus.cc:729] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Initialized, Role: FOLLOWER
I20260925 09:20:43.647775 11605 consensus_queue.cc:260] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.650450 11606 tablet_bootstrap.cc:492] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: No bootstrap required, opened a new log
I20260925 09:20:43.650749 11606 ts_tablet_manager.cc:1397] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260925 09:20:43.651249 11605 ts_tablet_manager.cc:1428] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:20:43.652064 11606 raft_consensus.cc:348] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.652478 11606 raft_consensus.cc:374] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:43.654529 11606 raft_consensus.cc:729] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32836bcc18a34ac784457e2922590f58, State: Initialized, Role: FOLLOWER
I20260925 09:20:43.655016 11606 consensus_queue.cc:260] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.655833 11606 ts_tablet_manager.cc:1428] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:20:43.660948 11604 tablet_bootstrap.cc:492] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Bootstrap starting.
I20260925 09:20:43.663093 11604 tablet_bootstrap.cc:654] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:43.666786 11604 tablet_bootstrap.cc:492] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: No bootstrap required, opened a new log
I20260925 09:20:43.667143 11604 ts_tablet_manager.cc:1397] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260925 09:20:43.668205 11604 raft_consensus.cc:348] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.668648 11604 raft_consensus.cc:374] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:43.668800 11604 raft_consensus.cc:729] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4508eaa827384d0cbf0150dda973a447, State: Initialized, Role: FOLLOWER
I20260925 09:20:43.669200 11604 consensus_queue.cc:260] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.670028 11604 ts_tablet_manager.cc:1428] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 09:20:43.967697 11537 raft_consensus.cc:482] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:20:43.967957 11537 raft_consensus.cc:504] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.968707 11537 leader_election.cc:288] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583), 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577)
I20260925 09:20:43.969350 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d908d6b9c0a438d8d3b910eaea4b756" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447" is_pre_election: true
I20260925 09:20:43.969455 11302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d908d6b9c0a438d8d3b910eaea4b756" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32836bcc18a34ac784457e2922590f58" is_pre_election: true
I20260925 09:20:43.969611 11162 raft_consensus.cc:2454] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 0.
I20260925 09:20:43.969799 11302 raft_consensus.cc:2454] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 0.
I20260925 09:20:43.970211 11372 leader_election.cc:302] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4508eaa827384d0cbf0150dda973a447, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:43.970552 11537 raft_consensus.cc:2781] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:20:43.970679 11537 raft_consensus.cc:482] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:20:43.970779 11537 raft_consensus.cc:3037] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:43.972914 11537 raft_consensus.cc:504] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.973596 11537 leader_election.cc:288] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 election: Requested vote from peers 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583), 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577)
I20260925 09:20:43.974224 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d908d6b9c0a438d8d3b910eaea4b756" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4508eaa827384d0cbf0150dda973a447"
I20260925 09:20:43.974262 11302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d908d6b9c0a438d8d3b910eaea4b756" candidate_uuid: "89d9c87e3619401c9f907fb38e8fee3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32836bcc18a34ac784457e2922590f58"
I20260925 09:20:43.974462 11162 raft_consensus.cc:3037] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:43.974550 11302 raft_consensus.cc:3037] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:43.977528 11162 raft_consensus.cc:2454] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 1.
I20260925 09:20:43.977905 11302 raft_consensus.cc:2454] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89d9c87e3619401c9f907fb38e8fee3a in term 1.
I20260925 09:20:43.978102 11372 leader_election.cc:302] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4508eaa827384d0cbf0150dda973a447, 89d9c87e3619401c9f907fb38e8fee3a; no voters:
I20260925 09:20:43.978526 11537 raft_consensus.cc:2781] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:43.979084 11537 raft_consensus.cc:686] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 LEADER]: Becoming Leader. State: Replica: 89d9c87e3619401c9f907fb38e8fee3a, State: Running, Role: LEADER
I20260925 09:20:43.979452 11537 consensus_queue.cc:237] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } }
I20260925 09:20:43.982550 11003 catalog_manager.cc:5495] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3). New cstate: current_term: 1 leader_uuid: "89d9c87e3619401c9f907fb38e8fee3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89d9c87e3619401c9f907fb38e8fee3a" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44991 } health_report { overall_health: HEALTHY } } }
I20260925 09:20:44.029349 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.022102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.3.82.3:44991)
0925 09:20:44.024330 (+ 2228us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:44.024334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:44.024355 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:20:44.024416 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:44.024420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:44.024485 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:44.024612 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:44.025123 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.025976 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.027408 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.027887 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.028030 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.028513 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:44.028631 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:44.028734 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:44.028891 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 09:20:44.028972 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2117,"thread_start_us":1931,"threads_started":1}
I20260925 09:20:44.047523 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.041450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50631 (local address 127.3.82.2:43577)
0925 09:20:44.041627 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:44.041632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:44.042382 (+ 750us) server_negotiation.cc:407] Connection header received
0925 09:20:44.042686 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:44.042691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:44.042777 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:44.042882 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:44.043409 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.044154 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.045576 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.046062 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.046210 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.046711 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:44.046828 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:44.046947 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:44.047099 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 09:20:44.047185 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:20:44.047648 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.041421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43577 (local address 127.3.82.3:50631)
0925 09:20:44.042258 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:44.042276 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:44.042470 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:44.042906 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:44.042911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:44.042923 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:20:44.043225 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:44.043235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.044304 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.044311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:44.045417 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:44.045430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.046275 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.046282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:44.046428 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.046495 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:20:44.046964 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:20:44.046977 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 09:20:44.047075 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 09:20:44.047236 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":511,"threads_started":1}
I20260925 09:20:44.056532 11162 raft_consensus.cc:1261] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Refusing update from remote peer 89d9c87e3619401c9f907fb38e8fee3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:44.057236 11302 raft_consensus.cc:1261] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Refusing update from remote peer 89d9c87e3619401c9f907fb38e8fee3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:20:44.057214 11620 consensus_queue.cc:1035] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4508eaa827384d0cbf0150dda973a447" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:44.057812 11537 consensus_queue.cc:1035] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32836bcc18a34ac784457e2922590f58" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:20:44.087273 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.080604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:33419 (local address 127.3.82.3:44991)
0925 09:20:44.080792 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:44.080796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:44.081467 (+ 671us) server_negotiation.cc:407] Connection header received
0925 09:20:44.081667 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:44.081672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:44.081777 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:44.081917 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:20:44.082545 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.083569 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.085097 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.085612 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.085745 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.086373 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:20:44.086493 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:20:44.086704 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:44.086905 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 09:20:44.086972 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 09:20:44.087410 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.080561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44991 (local address 127.3.82.2:33419)
0925 09:20:44.081314 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:44.081329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:44.081543 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:44.081944 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:44.081949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:44.081962 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 09:20:44.082328 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:44.082338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.083726 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.083731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:44.084905 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:44.084942 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.085875 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.085883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:44.086062 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.086122 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 09:20:44.086728 (+ 606us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 09:20:44.086739 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:20:44.086868 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 09:20:44.087081 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":438,"threads_started":1}
I20260925 09:20:44.116677 11005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:44.116912 11005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:20:44.120060 11005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a83bcb6b82a4086a1844e77da1b999e: Sending DeleteTablet for 3 replicas of tablet 3d908d6b9c0a438d8d3b910eaea4b756
I20260925 09:20:44.121135 11142 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d908d6b9c0a438d8d3b910eaea4b756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:44 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:39354
I20260925 09:20:44.121235 11281 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d908d6b9c0a438d8d3b910eaea4b756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:44 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:42292
I20260925 09:20:44.121234 11421 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d908d6b9c0a438d8d3b910eaea4b756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:20:44 UTC) from {username='oryx', principal='oryx/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:36624
I20260925 09:20:44.121853 11633 tablet_replica.cc:330] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: stopping tablet replica
I20260925 09:20:44.122067 11633 raft_consensus.cc:2229] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:44.122328 11633 raft_consensus.cc:2258] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:44.122509 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 11076
I20260925 09:20:44.123520 11633 ts_tablet_manager.cc:1905] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:44.133001 11633 ts_tablet_manager.cc:1918] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:20:44.133241 11633 log.cc:1196] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/3d908d6b9c0a438d8d3b910eaea4b756
I20260925 09:20:44.133882 11633 ts_tablet_manager.cc:1939] T 3d908d6b9c0a438d8d3b910eaea4b756 P 4508eaa827384d0cbf0150dda973a447: Deleting consensus metadata
I20260925 09:20:44.134222 11635 tablet_replica.cc:330] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: stopping tablet replica
I20260925 09:20:44.134631 11635 raft_consensus.cc:2229] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:20:44.135293 11635 raft_consensus.cc:2258] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:44.135390 10989 catalog_manager.cc:4841] TS 4508eaa827384d0cbf0150dda973a447 (127.3.82.1:35583): tablet 3d908d6b9c0a438d8d3b910eaea4b756 (table test-table [id=bc3f338fc4b04f36bb22f4e5847c2afc]) successfully deleted
I20260925 09:20:44.136118 11634 tablet_replica.cc:330] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: stopping tablet replica
I20260925 09:20:44.136426 11634 raft_consensus.cc:2229] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:20:44.136732 11634 raft_consensus.cc:2258] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:20:44.137585 11635 ts_tablet_manager.cc:1905] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:20:44.137970 11634 ts_tablet_manager.cc:1905] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 09:20:44 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.1@KRBTEST.COM
I20260925 09:20:44.145408 11634 ts_tablet_manager.cc:1918] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:20:44.145407 11635 ts_tablet_manager.cc:1918] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:20:44.145573 11635 log.cc:1196] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/3d908d6b9c0a438d8d3b910eaea4b756
I20260925 09:20:44.145573 11634 log.cc:1196] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/3d908d6b9c0a438d8d3b910eaea4b756
I20260925 09:20:44.146106 11634 ts_tablet_manager.cc:1939] T 3d908d6b9c0a438d8d3b910eaea4b756 P 32836bcc18a34ac784457e2922590f58: Deleting consensus metadata
I20260925 09:20:44.146117 11635 ts_tablet_manager.cc:1939] T 3d908d6b9c0a438d8d3b910eaea4b756 P 89d9c87e3619401c9f907fb38e8fee3a: Deleting consensus metadata
I20260925 09:20:44.147321 10989 catalog_manager.cc:4841] TS 32836bcc18a34ac784457e2922590f58 (127.3.82.2:43577): tablet 3d908d6b9c0a438d8d3b910eaea4b756 (table test-table [id=bc3f338fc4b04f36bb22f4e5847c2afc]) successfully deleted
I20260925 09:20:44.147428 10991 catalog_manager.cc:4841] TS 89d9c87e3619401c9f907fb38e8fee3a (127.3.82.3:44991): tablet 3d908d6b9c0a438d8d3b910eaea4b756 (table test-table [id=bc3f338fc4b04f36bb22f4e5847c2afc]) successfully deleted
I20260925 09:20:44.151590 11637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.124881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.3.82.1:35583)
0925 09:20:44.128235 (+ 3354us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:44.128245 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:44.135852 (+ 7607us) server_negotiation.cc:407] Connection header received
0925 09:20:44.136169 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:44.136178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:44.136281 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:44.136463 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:44.138257 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.139017 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.140281 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.140729 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.140868 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.145239 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:44.145257 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:44.145273 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:44.145312 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:44.148776 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:44.149509 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:44.149519 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:44.149521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:44.149689 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:44.150143 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:44.150147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:44.150149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:44.150527 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:44.150791 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:44.151091 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 09:20:44.151297 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3124,"thread_start_us":3026,"threads_started":1}
I20260925 09:20:44.152367 11207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:44.284312 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 11076
I20260925 09:20:44.310022 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 11216
Sep 25 09:20:44 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.2@KRBTEST.COM
I20260925 09:20:44.328156 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.311000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.3.82.2:43577)
0925 09:20:44.311225 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:44.311230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:44.311247 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:44.311310 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:44.311314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:44.311390 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:44.311486 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:44.312926 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.313769 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.315256 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.315743 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.315863 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.320890 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:44.320908 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:44.320926 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:44.320978 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:44.324784 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:44.325699 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:44.325709 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:44.325711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:44.325986 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:44.326542 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:44.326548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:44.326551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:44.327003 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:44.327230 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:44.327692 (+ 462us) server_negotiation.cc:299] Negotiation successful
0925 09:20:44.327875 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:20:44.328954 11347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:44.482652 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 11216
I20260925 09:20:44.508718 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 11356
Sep 25 09:20:44 dist-test-slave-4wm5 krb5kdc[10960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.82.3@KRBTEST.COM
I20260925 09:20:44.526289 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:44.509666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.3.82.3:44991)
0925 09:20:44.509935 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:44.509940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:44.509978 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:20:44.510133 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:44.510137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:44.510201 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:44.510291 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:44.511740 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.512614 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.514088 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:44.514647 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:44.514767 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:44.519633 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:44.519650 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:44.519667 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:44.519715 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:44.523009 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:44.523895 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:44.523904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:44.523906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:44.524065 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:44.524498 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:44.524521 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:44.524524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:44.524913 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:44.525268 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:44.525599 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 09:20:44.525888 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:20:44.527130 11487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:44.655040 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 11356
I20260925 09:20:44.677421 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 10973
I20260925 09:20:44.678823 11045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:44.798959 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 10973
2026-09-25T09:20:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6712 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:44 dist-test-slave-4wm5 krb5kdc[11644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:44 dist-test-slave-4wm5 krb5kdc[11644](info): set up 2 sockets
Sep 25 09:20:44 dist-test-slave-4wm5 krb5kdc[11644](info): commencing operation
krb5kdc: starting...
W20260925 09:20:45.995078 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:46 dist-test-slave-4wm5 krb5kdc[11644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:46.243214 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:46007
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:46007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:46.594484 11657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:46.594869 11657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:46.595021 11657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:46.603092 11657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:46.603291 11657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:46.603410 11657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:46.603521 11657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:46.603610 11657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:46.614507 11657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:46007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:46007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:46.615971 11657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:46.617480 11657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:46.630927 11662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:46.630921 11665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:46.630879 11663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:46.632432 11657 server_base.cc:1034] running on GCE node
I20260925 09:20:46.633404 11657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:46.635140 11657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:46.636421 11657 hybrid_clock.cc:648] HybridClock initialized: now 1790328046636367 us; error 48 us; skew 500 ppm
I20260925 09:20:46.636909 11657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T09:20:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1903 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:20:46 dist-test-slave-4wm5 krb5kdc[11671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:20:46 dist-test-slave-4wm5 krb5kdc[11671](info): set up 2 sockets
Sep 25 09:20:46 dist-test-slave-4wm5 krb5kdc[11671](info): commencing operation
krb5kdc: starting...
W20260925 09:20:47.846825 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:20:48 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:20:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:20:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:20:48.097940 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36655
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:48.456012 11684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:48.456351 11684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:48.456449 11684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:48.464124 11684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:48.464269 11684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:48.464322 11684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:48.464385 11684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:48.464434 11684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:48.475010 11684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:48.476531 11684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:48.477941 11684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:48.490808 11689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:48.490754 11692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:48.490839 11690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:48.491564 11684 server_base.cc:1034] running on GCE node
I20260925 09:20:48.492409 11684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:48.493943 11684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:48.495229 11684 hybrid_clock.cc:648] HybridClock initialized: now 1790328048495132 us; error 58 us; skew 500 ppm
I20260925 09:20:48.495612 11684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:48 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:48.499089 11684 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:48.500815 11684 webserver.cc:460] Webserver started at http://127.3.82.62:40423/ using document root <none> and password file <none>
I20260925 09:20:48.501521 11684 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:48.501626 11684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:48.501998 11684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:48.505198 11684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "cdb5190d2d57431e82616546a0f521e3"
format_stamp: "Formatted at 2026-09-25 09:20:48 on dist-test-slave-4wm5"
server_key: "13235f1a21aa04bafa21a00233a80709"
server_key_iv: "77ebc9a914d79482dab367ec60ae2112"
server_key_version: "encryptionkey@0"
I20260925 09:20:48.506093 11684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "cdb5190d2d57431e82616546a0f521e3"
format_stamp: "Formatted at 2026-09-25 09:20:48 on dist-test-slave-4wm5"
server_key: "13235f1a21aa04bafa21a00233a80709"
server_key_iv: "77ebc9a914d79482dab367ec60ae2112"
server_key_version: "encryptionkey@0"
I20260925 09:20:48.511910 11684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:20:48.515740 11699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:48.517235 11684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:20:48.517390 11684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "cdb5190d2d57431e82616546a0f521e3"
format_stamp: "Formatted at 2026-09-25 09:20:48 on dist-test-slave-4wm5"
server_key: "13235f1a21aa04bafa21a00233a80709"
server_key_iv: "77ebc9a914d79482dab367ec60ae2112"
server_key_version: "encryptionkey@0"
I20260925 09:20:48.517566 11684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:48.541704 11684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:48.546250 11684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:48.546497 11684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:48.565758 11684 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36655
I20260925 09:20:48.565817 11750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36655 every 8 connection(s)
I20260925 09:20:48.567310 11684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:48.571401 11751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:20:48.572562 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 11684
I20260925 09:20:48.572778 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:20:48.573614 3400 external_mini_cluster.cc:1428] Setting key 390975300b802e90d00b8a2819822d23
I20260925 09:20:48.587985 11751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Bootstrap starting.
Sep 25 09:20:48 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:48.592178 11751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Neither blocks nor log segments found. Creating new log.
I20260925 09:20:48.593438 11751 log.cc:824] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:48.596903 11751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: No bootstrap required, opened a new log
I20260925 09:20:48.599362 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:48.575988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.3.82.62:36655)
0925 09:20:48.576924 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:48.576938 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:48.577003 (+ 65us) server_negotiation.cc:407] Connection header received
0925 09:20:48.578828 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:48.578858 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:48.579337 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:48.579727 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:48.582070 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:48.583459 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:48.585576 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:48.586404 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:48.586669 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:48.591629 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:48.591638 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:48.591655 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:48.591703 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:48.595149 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:48.595919 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:48.595926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:48.595933 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:48.596100 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:48.596494 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:48.596500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:48.596503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:48.597059 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:48.597381 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:48.597885 (+ 504us) server_negotiation.cc:299] Negotiation successful
0925 09:20:48.598314 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":439,"threads_started":1}
I20260925 09:20:48.605052 11751 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:48.605472 11751 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:20:48.605613 11751 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb5190d2d57431e82616546a0f521e3, State: Initialized, Role: FOLLOWER
I20260925 09:20:48.606290 11751 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:48.606534 11751 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:48.606679 11751 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:48.606851 11751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:20:48.609326 11751 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:48.609915 11751 leader_election.cc:302] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdb5190d2d57431e82616546a0f521e3; no voters:
I20260925 09:20:48.610498 11751 leader_election.cc:288] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:20:48.610649 11756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:20:48.611367 11756 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 LEADER]: Becoming Leader. State: Replica: cdb5190d2d57431e82616546a0f521e3, State: Running, Role: LEADER
I20260925 09:20:48.611891 11756 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:48.612654 11751 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:48.614778 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdb5190d2d57431e82616546a0f521e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } } }
I20260925 09:20:48.615249 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:48.615859 11758 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdb5190d2d57431e82616546a0f521e3. Latest consensus state: current_term: 1 leader_uuid: "cdb5190d2d57431e82616546a0f521e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } } }
I20260925 09:20:48.616251 11758 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:48.616962 11762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:48.622493 11762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:48.633000 11762 catalog_manager.cc:1347] Generated new cluster ID: 743e3b015f8a45559230d245a76e4a7f
I20260925 09:20:48.633186 11762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:48.651360 11762 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:20:48.652843 11762 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:48.661631 11762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Generated new TSK 0
I20260925 09:20:48.662606 11762 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:20:48.734889 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36655
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:49.094172 11777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:49.094483 11777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:49.094583 11777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:49.102277 11777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:49.102396 11777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:49.102519 11777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:49.112702 11777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:49.114138 11777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:49.115561 11777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:49.128839 11782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:49.129971 11783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:49.131260 11785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:49.131577 11777 server_base.cc:1034] running on GCE node
I20260925 09:20:49.132257 11777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:49.133369 11777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:49.134604 11777 hybrid_clock.cc:648] HybridClock initialized: now 1790328049134564 us; error 73 us; skew 500 ppm
I20260925 09:20:49.135004 11777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:49 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328049, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:49.138324 11777 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:20:49.139858 11777 webserver.cc:460] Webserver started at http://127.3.82.1:40607/ using document root <none> and password file <none>
I20260925 09:20:49.140533 11777 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:49.140637 11777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:49.140928 11777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:49.145648 11777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "410b3dfeb88146b6ad9e6f05415c315e"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "b315f61bc990b89effcd090f3a20ac60"
server_key_iv: "865d5dd36d191c59c9ad3249cca2f9d3"
server_key_version: "encryptionkey@0"
I20260925 09:20:49.146744 11777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "410b3dfeb88146b6ad9e6f05415c315e"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "b315f61bc990b89effcd090f3a20ac60"
server_key_iv: "865d5dd36d191c59c9ad3249cca2f9d3"
server_key_version: "encryptionkey@0"
I20260925 09:20:49.153023 11777 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:20:49.156999 11792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:49.158694 11777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:20:49.158907 11777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "410b3dfeb88146b6ad9e6f05415c315e"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "b315f61bc990b89effcd090f3a20ac60"
server_key_iv: "865d5dd36d191c59c9ad3249cca2f9d3"
server_key_version: "encryptionkey@0"
I20260925 09:20:49.159130 11777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:49.181810 11777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:49.186555 11777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:49.186910 11777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:49.187968 11777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:49.189432 11777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:49.189541 11777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:49.189693 11777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:49.189764 11777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:49.224679 11777 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:34077
I20260925 09:20:49.224686 11904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:34077 every 8 connection(s)
I20260925 09:20:49.226181 11777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:49.229421 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 11777
I20260925 09:20:49.229635 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:20:49.230326 3400 external_mini_cluster.cc:1428] Setting key 993fdc31e3ba92b4d5e72325100a864a
Sep 25 09:20:49 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328049, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:49.251005 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:49.229347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:42463 (local address 127.3.82.62:36655)
0925 09:20:49.232792 (+ 3445us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:49.232800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:49.232822 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:20:49.233942 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:49.233948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:49.234085 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:49.234207 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:49.235949 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.236690 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.238468 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.239059 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.239217 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:49.243197 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:49.243207 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:49.243214 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:49.243267 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:49.247107 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:49.248189 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:49.248195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:49.248197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:49.248340 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:49.248944 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:49.248952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:49.248957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:49.249271 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:49.249445 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:49.250397 (+ 952us) server_negotiation.cc:299] Negotiation successful
0925 09:20:49.250644 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3224,"thread_start_us":3092,"threads_started":1}
I20260925 09:20:49.252126 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:49.230145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:42463)
0925 09:20:49.231205 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:49.231262 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:49.233604 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:49.234469 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:49.234478 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:49.235124 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:49.235749 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:49.235768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.236870 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.236874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:49.238306 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:49.238325 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.239321 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.239327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:49.239633 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:49.240484 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:49.240509 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:49.242905 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:49.247327 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:49.247337 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:49.247353 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:49.247977 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:49.248470 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:49.248475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:49.248484 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:49.248734 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:49.249425 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:49.249434 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:49.249989 (+ 555us) client_negotiation.cc:769] Sending connection context
0925 09:20:49.250280 (+ 291us) client_negotiation.cc:240] Negotiation successful
0925 09:20:49.250925 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":431,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:20:49.253966 11905 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36655
I20260925 09:20:49.254268 11905 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:49.254978 11905 heartbeater.cc:511] Master 127.3.82.62:36655 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:20:49.257534 11716 ts_manager.cc:194] Registered new tserver with Master: 410b3dfeb88146b6ad9e6f05415c315e (127.3.82.1:34077)
I20260925 09:20:49.259806 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:42463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:20:49.286145 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36655
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:49.632669 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:49.633049 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:49.633152 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:49.641108 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:49.641240 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:49.641357 11911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:49.651690 11911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:49.652997 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:49.654393 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:49.668869 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:49.669122 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:49.669520 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:49.670162 11911 server_base.cc:1034] running on GCE node
I20260925 09:20:49.670809 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:49.671936 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:49.673152 11911 hybrid_clock.cc:648] HybridClock initialized: now 1790328049673084 us; error 64 us; skew 500 ppm
I20260925 09:20:49.673501 11911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:49 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328049, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:49.676962 11911 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:20:49.678620 11911 webserver.cc:460] Webserver started at http://127.3.82.2:33841/ using document root <none> and password file <none>
I20260925 09:20:49.679392 11911 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:49.679500 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:49.679771 11911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:49.682938 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "57bebd1d71ad446f9c998929db9f5419"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "e3a3ab1f58a1bd80d0035cfb1c80cb5b"
server_key_iv: "6d4475fa874d598c693ad7345432133c"
server_key_version: "encryptionkey@0"
I20260925 09:20:49.683712 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "57bebd1d71ad446f9c998929db9f5419"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "e3a3ab1f58a1bd80d0035cfb1c80cb5b"
server_key_iv: "6d4475fa874d598c693ad7345432133c"
server_key_version: "encryptionkey@0"
I20260925 09:20:49.689522 11911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:20:49.693246 11926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:49.694797 11911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:49.694959 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "57bebd1d71ad446f9c998929db9f5419"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "e3a3ab1f58a1bd80d0035cfb1c80cb5b"
server_key_iv: "6d4475fa874d598c693ad7345432133c"
server_key_version: "encryptionkey@0"
I20260925 09:20:49.695231 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:49.712702 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:49.717026 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:49.717336 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:49.718494 11911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:49.719902 11911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:49.719980 11911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:49.720129 11911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:49.720178 11911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:49.754417 11911 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:36239
I20260925 09:20:49.754475 12038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:36239 every 8 connection(s)
I20260925 09:20:49.755806 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:49.759796 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 11911
I20260925 09:20:49.759989 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:20:49.760571 3400 external_mini_cluster.cc:1428] Setting key c9898135728b97aafa2976d136aae171
Sep 25 09:20:49 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328049, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:49.781800 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:49.758622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:50119 (local address 127.3.82.62:36655)
0925 09:20:49.759114 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:49.759119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:49.760225 (+ 1106us) server_negotiation.cc:407] Connection header received
0925 09:20:49.762724 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:49.762729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:49.762814 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:49.762940 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:49.765372 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.766256 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.768988 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.769462 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.769592 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:49.773683 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:49.773693 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:49.773700 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:49.773753 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:49.777665 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:49.778797 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:49.778805 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:49.778812 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:49.779012 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:49.779552 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:49.779560 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:49.779566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:49.779993 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:49.780175 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:49.780959 (+ 784us) server_negotiation.cc:299] Negotiation successful
0925 09:20:49.781269 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":261,"threads_started":1}
I20260925 09:20:49.782637 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:49.759047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:50119)
0925 09:20:49.759959 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:49.760022 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:49.762368 (+ 2346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:49.763336 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:49.763349 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:49.764237 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:49.765149 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:49.765179 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.766435 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.766443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:49.768712 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:49.768762 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:49.769720 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:49.769729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:49.770124 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:49.770993 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:49.771031 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:49.773374 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:49.777982 (+ 4608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:49.777991 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:49.778007 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:49.778613 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:49.779166 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:49.779170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:49.779172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:49.779409 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:49.780140 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:49.780147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:49.780532 (+ 385us) client_negotiation.cc:769] Sending connection context
0925 09:20:49.780799 (+ 267us) client_negotiation.cc:240] Negotiation successful
0925 09:20:49.781257 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":338,"threads_started":1}
I20260925 09:20:49.784360 12039 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36655
I20260925 09:20:49.784734 12039 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:49.785773 12039 heartbeater.cc:511] Master 127.3.82.62:36655 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:20:49.787745 11716 ts_manager.cc:194] Registered new tserver with Master: 57bebd1d71ad446f9c998929db9f5419 (127.3.82.2:36239)
I20260925 09:20:49.788784 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:50119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:20:49.821929 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36655
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:50.164973 12045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:50.165335 12045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:50.165503 12045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:50.173249 12045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:50.173439 12045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:50.173625 12045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:50.184108 12045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:50.185536 12045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:50.186941 12045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:50.201109 12050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:50.201188 12051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:50.201447 12053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:50.202294 12045 server_base.cc:1034] running on GCE node
I20260925 09:20:50.203034 12045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:50.204115 12045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:50.205387 12045 hybrid_clock.cc:648] HybridClock initialized: now 1790328050205336 us; error 42 us; skew 500 ppm
I20260925 09:20:50.205817 12045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:50 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328050, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:50.216197 12045 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:20:50.218183 12045 webserver.cc:460] Webserver started at http://127.3.82.3:32987/ using document root <none> and password file <none>
I20260925 09:20:50.219007 12045 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:50.219164 12045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:50.219545 12045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:20:50.222802 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "661c8d8d40a449fa868a65ba55a8b7cb"
format_stamp: "Formatted at 2026-09-25 09:20:50 on dist-test-slave-4wm5"
server_key: "3a16f6bba6a449bbd6bbe589efe5622a"
server_key_iv: "4c424be19b449c369d8048e31a19e2b1"
server_key_version: "encryptionkey@0"
I20260925 09:20:50.223753 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "661c8d8d40a449fa868a65ba55a8b7cb"
format_stamp: "Formatted at 2026-09-25 09:20:50 on dist-test-slave-4wm5"
server_key: "3a16f6bba6a449bbd6bbe589efe5622a"
server_key_iv: "4c424be19b449c369d8048e31a19e2b1"
server_key_version: "encryptionkey@0"
I20260925 09:20:50.229753 12045 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260925 09:20:50.233789 12060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:50.235420 12045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:50.235638 12045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "661c8d8d40a449fa868a65ba55a8b7cb"
format_stamp: "Formatted at 2026-09-25 09:20:50 on dist-test-slave-4wm5"
server_key: "3a16f6bba6a449bbd6bbe589efe5622a"
server_key_iv: "4c424be19b449c369d8048e31a19e2b1"
server_key_version: "encryptionkey@0"
I20260925 09:20:50.235869 12045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:50.258339 12045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:50.263546 12045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:50.263545 11905 heartbeater.cc:503] Master 127.3.82.62:36655 was elected leader, sending a full tablet report...
I20260925 09:20:50.263864 12045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:50.265139 12045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:50.266755 12045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:50.266829 12045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:50.266921 12045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:50.266973 12045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:50.301931 12045 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:44017
I20260925 09:20:50.301968 12172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:44017 every 8 connection(s)
I20260925 09:20:50.303257 12045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:50.306229 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12045
I20260925 09:20:50.306449 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:20:50.306897 3400 external_mini_cluster.cc:1428] Setting key 103cdc918c8e6391fc91cfa3c5cf4800
Sep 25 09:20:50 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328050, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:20:50.326506 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:50.306099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:59041 (local address 127.3.82.62:36655)
0925 09:20:50.306676 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:50.306683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:50.307882 (+ 1199us) server_negotiation.cc:407] Connection header received
0925 09:20:50.310286 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:50.310290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:50.310358 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:50.310447 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:50.312148 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.312953 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.314791 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.315277 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.315390 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:50.319118 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:50.319125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:50.319129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:50.319190 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:50.322880 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.323803 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.323809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.323812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.323939 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.324429 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.324439 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.324442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.324744 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:50.324910 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:50.325722 (+ 812us) server_negotiation.cc:299] Negotiation successful
0925 09:20:50.326024 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":293,"threads_started":1}
I20260925 09:20:50.327690 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:50.306827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:59041)
0925 09:20:50.307693 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:50.307752 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:50.309975 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:50.310673 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:50.310681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:50.311292 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:50.311959 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:50.311978 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.313141 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.313145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:50.314623 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:50.314636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.315495 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.315500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:50.315701 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:50.316386 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:50.316412 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:50.318871 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:20:50.323059 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:50.323071 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:50.323093 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:50.323642 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:50.324064 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:20:50.324069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:20:50.324073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:20:50.324298 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:20:50.324885 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:20:50.324895 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:20:50.325307 (+ 412us) client_negotiation.cc:769] Sending connection context
0925 09:20:50.325602 (+ 295us) client_negotiation.cc:240] Negotiation successful
0925 09:20:50.326289 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":335,"threads_started":1}
I20260925 09:20:50.329288 12173 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36655
I20260925 09:20:50.329638 12173 heartbeater.cc:464] Registering TS with master...
I20260925 09:20:50.330523 12173 heartbeater.cc:511] Master 127.3.82.62:36655 requested a full tablet report, sending...
I20260925 09:20:50.332201 11716 ts_manager.cc:194] Registered new tserver with Master: 661c8d8d40a449fa868a65ba55a8b7cb (127.3.82.3:44017)
I20260925 09:20:50.333216 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:59041
I20260925 09:20:50.336041 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab.
Entry for principal oryx/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab.
WARNING: no policy specified for oryx/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab.
Entry for principal oryx/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab.
WARNING: no policy specified for oryx/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab.
Entry for principal oryx/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab.
I20260925 09:20:50.489265 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 11777
Sep 25 09:20:50 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:20:50.506855 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:50.490697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.3.82.1:34077)
0925 09:20:50.491324 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:50.491331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:50.491360 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:20:50.491424 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:50.491431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:50.491720 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:50.491869 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:50.493168 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.494329 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.495669 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.496226 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.496325 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:50.500858 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:50.500872 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:50.500888 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:50.500934 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:50.504016 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.504753 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.504761 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.504763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.504901 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.505230 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.505233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.505235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.505570 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:50.505800 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:50.506173 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 09:20:50.506390 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":325,"threads_started":1}
I20260925 09:20:50.508231 11900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:50.593585 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 11777
I20260925 09:20:50.610303 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 11911
Sep 25 09:20:50 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:20:50.628610 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:50.611375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.3.82.2:36239)
0925 09:20:50.611904 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:50.611914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:50.611949 (+ 35us) server_negotiation.cc:407] Connection header received
0925 09:20:50.612025 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:50.612031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:50.612257 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:50.612432 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:50.613860 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.614870 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.616177 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.616737 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.616882 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:50.621474 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:50.621491 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:50.621514 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:50.621569 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:50.624874 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.625689 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.625697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.625699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.626032 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.626445 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.626449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.626451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.626920 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:50.627162 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:50.627660 (+ 498us) server_negotiation.cc:299] Negotiation successful
0925 09:20:50.627992 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":293,"threads_started":1}
I20260925 09:20:50.630040 12034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:50.714531 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 11911
I20260925 09:20:50.730912 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12045
Sep 25 09:20:50 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:20:50.749089 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:50.732028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.3.82.3:44017)
0925 09:20:50.732848 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:50.732855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:50.732903 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:20:50.732990 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:50.733001 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:50.733224 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:50.733380 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:50.734915 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.735982 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.737375 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:50.738067 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:50.738174 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:50.742931 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:50.742945 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:50.742960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:50.743017 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:50.746089 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.746839 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.746846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.746849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.746997 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:50.747393 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:50.747398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:50.747402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:50.747782 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:50.748130 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:50.748439 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 09:20:50.748729 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":409,"threads_started":1}
I20260925 09:20:50.750507 12168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:50.835214 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12045
I20260925 09:20:50.852666 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 11684
I20260925 09:20:50.853995 11746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:20:50.960062 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 11684
I20260925 09:20:50.981413 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36655
--webserver_interface=127.3.82.62
--webserver_port=40423
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:20:51.350610 12191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:51.350991 12191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:51.351176 12191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:51.359848 12191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:20:51.359993 12191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:51.360112 12191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:51.360201 12191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:20:51.360278 12191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:20:51.371054 12191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=40423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:51.372543 12191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:51.374079 12191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:51.387506 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:51.387491 12197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:51.387423 12196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:51.388573 12191 server_base.cc:1034] running on GCE node
I20260925 09:20:51.389606 12191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:51.391791 12191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:51.393044 12191 hybrid_clock.cc:648] HybridClock initialized: now 1790328051392970 us; error 70 us; skew 500 ppm
I20260925 09:20:51.393399 12191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:51 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328051, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:51.396857 12191 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:20:51.398516 12191 webserver.cc:460] Webserver started at http://127.3.82.62:40423/ using document root <none> and password file <none>
I20260925 09:20:51.399241 12191 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:51.399361 12191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:51.405305 12191 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 09:20:51.408774 12206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:51.410596 12191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:20:51.410825 12191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "cdb5190d2d57431e82616546a0f521e3"
format_stamp: "Formatted at 2026-09-25 09:20:48 on dist-test-slave-4wm5"
server_key: "13235f1a21aa04bafa21a00233a80709"
server_key_iv: "77ebc9a914d79482dab367ec60ae2112"
server_key_version: "encryptionkey@0"
I20260925 09:20:51.411916 12191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:51.439913 12191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:51.444957 12191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:51.445312 12191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:51.465494 12191 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36655
I20260925 09:20:51.465533 12257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36655 every 8 connection(s)
I20260925 09:20:51.467211 12191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:20:51.475096 12258 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:20:51.476749 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12191
I20260925 09:20:51.477736 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:34077
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:36655
--webserver_port=40607
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:20:51.479849 12258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Bootstrap starting.
I20260925 09:20:51.501533 12258 log.cc:824] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:20:51.511662 12258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:20:51.512228 12258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Bootstrap complete.
I20260925 09:20:51.522184 12258 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:51.523082 12258 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb5190d2d57431e82616546a0f521e3, State: Initialized, Role: FOLLOWER
I20260925 09:20:51.523833 12258 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:51.524080 12258 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:20:51.524199 12258 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:20:51.524379 12258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:20:51.528528 12258 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:51.529071 12258 leader_election.cc:302] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdb5190d2d57431e82616546a0f521e3; no voters:
I20260925 09:20:51.529757 12258 leader_election.cc:288] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:20:51.529929 12262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:20:51.531206 12262 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [term 2 LEADER]: Becoming Leader. State: Replica: cdb5190d2d57431e82616546a0f521e3, State: Running, Role: LEADER
I20260925 09:20:51.531793 12262 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } }
I20260925 09:20:51.532194 12258 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:20:51.534454 12264 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdb5190d2d57431e82616546a0f521e3. Latest consensus state: current_term: 2 leader_uuid: "cdb5190d2d57431e82616546a0f521e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } } }
I20260925 09:20:51.534816 12264 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:51.535300 12263 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cdb5190d2d57431e82616546a0f521e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb5190d2d57431e82616546a0f521e3" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36655 } } }
I20260925 09:20:51.535631 12263 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:20:51.538674 12268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:20:51.544759 12268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:20:51.546391 12268 catalog_manager.cc:1259] Loaded cluster ID: 743e3b015f8a45559230d245a76e4a7f
I20260925 09:20:51.546517 12268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:20:51.550755 12268 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:20:51.552260 12268 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cdb5190d2d57431e82616546a0f521e3: Loaded TSK: 0
I20260925 09:20:51.553848 12268 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:20:51.847406 12260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:51.847724 12260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:51.847878 12260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:51.856012 12260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:51.856154 12260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:51.856333 12260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:20:51.866878 12260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.1.keytab
--principal=oryx/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:34077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=40607
--tserver_master_addrs=127.3.82.62:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:51.868330 12260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:51.869686 12260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:51.886732 12284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:51.886765 12285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:51.887212 12287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:51.887967 12260 server_base.cc:1034] running on GCE node
I20260925 09:20:51.888756 12260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:51.889953 12260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:51.891232 12260 hybrid_clock.cc:648] HybridClock initialized: now 1790328051891167 us; error 57 us; skew 500 ppm
I20260925 09:20:51.891698 12260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:51 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328051, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:51.895839 12260 init.cc:377] Logged in from keytab as oryx/127.3.82.1@KRBTEST.COM (short username oryx)
I20260925 09:20:51.897770 12260 webserver.cc:460] Webserver started at http://127.3.82.1:40607/ using document root <none> and password file <none>
I20260925 09:20:51.898695 12260 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:51.898861 12260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:51.905632 12260 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.004s
I20260925 09:20:51.909435 12294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:51.911585 12260 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:20:51.911834 12260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "410b3dfeb88146b6ad9e6f05415c315e"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "b315f61bc990b89effcd090f3a20ac60"
server_key_iv: "865d5dd36d191c59c9ad3249cca2f9d3"
server_key_version: "encryptionkey@0"
I20260925 09:20:51.912967 12260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:51.951532 12260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:51.956431 12260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:51.956753 12260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:51.957792 12260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:51.959277 12260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:51.959375 12260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:51.959527 12260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:51.959623 12260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:51.994876 12260 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:34077
I20260925 09:20:51.994997 12406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:34077 every 8 connection(s)
I20260925 09:20:51.996428 12260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:20:52.006147 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12260
I20260925 09:20:52.006991 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:36239
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:36655
--webserver_port=33841
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:52.018162 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:51.999765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:52073)
0925 09:20:52.000614 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:52.000662 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:52.002615 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:52.004188 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:52.004197 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:52.004808 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:52.005422 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.005442 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.008291 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.008297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.009925 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.009941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.011512 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.011518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.011759 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.012500 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:52.012557 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:52.016658 (+ 4101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":382,"threads_started":1}
W20260925 09:20:52.018209 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:51.999677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:52073 (local address 127.3.82.62:36655)
0925 09:20:52.000425 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:52.000438 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:52.000825 (+ 387us) server_negotiation.cc:407] Connection header received
0925 09:20:52.003268 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:52.003285 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:52.003683 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:52.004022 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:52.005671 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.008041 (+ 2370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.010100 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.011253 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.011469 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.017050 (+ 5581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:52073: BlockingRecv error: recv got EOF from 127.3.82.1:52073 (error 108)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":304,"threads_started":1}
W20260925 09:20:52.019311 12407 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:36655 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.82.62:36655: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:52.029091 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:52.020083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:55225)
0925 09:20:52.020271 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:52.020292 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:52.020398 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:52.020820 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:52.020824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:52.021189 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:52.021353 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.021363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.022932 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.022937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.023947 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.023960 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.025039 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.025043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.025191 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.025928 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:52.025948 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:52.028867 (+ 2919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260925 09:20:52.029532 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:52.020221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:55225 (local address 127.3.82.62:36655)
0925 09:20:52.020424 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:52.020432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:52.020457 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:20:52.020533 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:52.020542 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:52.020628 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:52.020738 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:52.021528 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.022760 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.024101 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.024880 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.025041 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.029212 (+ 4171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:55225: BlockingRecv error: recv got EOF from 127.3.82.1:55225 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:52.039973 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:52.030075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:33147)
0925 09:20:52.030246 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:52.030268 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:52.030379 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:52.030792 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:52.030796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:52.031110 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:52.031270 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.031278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.032369 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.032375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.033504 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.033520 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.034558 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.034564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.034735 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.035435 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:52.035454 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:52.039609 (+ 4155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 09:20:52.040243 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:52.030298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:33147 (local address 127.3.82.62:36655)
0925 09:20:52.030491 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:52.030496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:52.030513 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:52.030573 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:52.030577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:52.030655 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:52.030760 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:52.031407 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.032232 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.033671 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.034379 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.034662 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.039864 (+ 5202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:33147: BlockingRecv error: recv got EOF from 127.3.82.1:33147 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 09:20:52.040465 12407 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:20:52.377185 12411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:52.377489 12411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:52.377581 12411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:52.384912 12411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:52.385033 12411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:52.385151 12411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:20:52.396508 12411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.2.keytab
--principal=oryx/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:36239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=33841
--tserver_master_addrs=127.3.82.62:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:52.397958 12411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:52.399264 12411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:52.412761 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:52.412839 12416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:52.412793 12417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:52.415870 12411 server_base.cc:1034] running on GCE node
I20260925 09:20:52.416715 12411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:52.417936 12411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:52.419195 12411 hybrid_clock.cc:648] HybridClock initialized: now 1790328052419122 us; error 65 us; skew 500 ppm
I20260925 09:20:52.419536 12411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328052, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:52.423102 12411 init.cc:377] Logged in from keytab as oryx/127.3.82.2@KRBTEST.COM (short username oryx)
I20260925 09:20:52.424800 12411 webserver.cc:460] Webserver started at http://127.3.82.2:33841/ using document root <none> and password file <none>
I20260925 09:20:52.425532 12411 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:52.425648 12411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:52.431721 12411 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:20:52.435052 12426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:52.436595 12411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:20:52.436820 12411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "57bebd1d71ad446f9c998929db9f5419"
format_stamp: "Formatted at 2026-09-25 09:20:49 on dist-test-slave-4wm5"
server_key: "e3a3ab1f58a1bd80d0035cfb1c80cb5b"
server_key_iv: "6d4475fa874d598c693ad7345432133c"
server_key_version: "encryptionkey@0"
I20260925 09:20:52.437896 12411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:52.453445 12411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:52.458532 12411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:52.458914 12411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:52.460064 12411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:52.461479 12411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:52.461572 12411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:52.461730 12411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:52.461869 12411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:52.498590 12411 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:36239
I20260925 09:20:52.498636 12538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:36239 every 8 connection(s)
I20260925 09:20:52.500051 12411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:20:52.505482 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12411
I20260925 09:20:52.506206 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:44017
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:36655
--webserver_port=32987
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:20:52.518458 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:52.502882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41783 (local address 127.3.82.62:36655)
0925 09:20:52.503105 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:52.503112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:52.504462 (+ 1350us) server_negotiation.cc:407] Connection header received
0925 09:20:52.506853 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:52.506859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:52.506980 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:52.507118 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:52.508830 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.509609 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.511518 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.512049 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.512222 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.518180 (+ 5958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:41783: BlockingRecv error: recv got EOF from 127.3.82.2:41783 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:20:52.519322 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:52.503443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:41783)
0925 09:20:52.504275 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:52.504322 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:52.506523 (+ 2201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:52.507389 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:52.507399 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:52.507974 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:52.508618 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.508638 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.509799 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.509804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.511322 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.511340 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.512337 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.512345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.512603 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.513393 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:52.513419 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:52.518064 (+ 4645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":356,"threads_started":1}
W20260925 09:20:52.520069 12539 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:36655 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.82.62:36655: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:52.529808 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:52.520797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:55015)
0925 09:20:52.521110 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:52.521126 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:52.521295 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:52.521709 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:52.521714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:52.522148 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:52.522299 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.522307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.523322 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.523327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.524266 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.524278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.525038 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.525044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.525225 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.525894 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:52.525913 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:52.529571 (+ 3658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:20:52.530606 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:52.520886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:55015 (local address 127.3.82.62:36655)
0925 09:20:52.521041 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:52.521046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:52.521220 (+ 174us) server_negotiation.cc:407] Connection header received
0925 09:20:52.521443 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:52.521447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:52.521511 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:52.521605 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:52.522450 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.523180 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.524415 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.524867 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.525040 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.529964 (+ 4924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:55015: BlockingRecv error: recv got EOF from 127.3.82.2:55015 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:52.541417 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:52.530679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:37301)
0925 09:20:52.531161 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:52.531182 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:52.531311 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:52.531762 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:52.531766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:52.532072 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:52.532240 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.532248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.533964 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.533968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.534939 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:52.534953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.535977 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.535981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:52.536112 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.536768 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:52.536787 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:52.541059 (+ 4272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342}
W20260925 09:20:52.541520 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:52.530766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:37301 (local address 127.3.82.62:36655)
0925 09:20:52.530972 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:52.530979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:52.531421 (+ 442us) server_negotiation.cc:407] Connection header received
0925 09:20:52.531492 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:52.531498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:52.531577 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:52.531682 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:52.532389 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.533748 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.535088 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:52.535823 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:52.535960 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:52.541292 (+ 5332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:37301: BlockingRecv error: recv got EOF from 127.3.82.2:37301 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":395,"threads_started":1}
W20260925 09:20:52.542075 12539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:20:52.858443 12542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:20:52.858820 12542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:20:52.858992 12542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:20:52.866582 12542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:20:52.866941 12542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:20:52.867161 12542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:20:52.877674 12542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/krb5kdc/oryx_127.3.82.3.keytab
--principal=oryx/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:44017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=32987
--tserver_master_addrs=127.3.82.62:36655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:20:52.879115 12542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:20:52.880534 12542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:20:52.894428 12549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:52.894403 12548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:20:52.894446 12551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:20:52.895673 12542 server_base.cc:1034] running on GCE node
I20260925 09:20:52.896413 12542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:20:52.897485 12542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:20:52.898751 12542 hybrid_clock.cc:648] HybridClock initialized: now 1790328052898681 us; error 36 us; skew 500 ppm
I20260925 09:20:52.899132 12542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:20:52 dist-test-slave-4wm5 krb5kdc[11671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328052, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:20:52.902406 12542 init.cc:377] Logged in from keytab as oryx/127.3.82.3@KRBTEST.COM (short username oryx)
I20260925 09:20:52.904156 12542 webserver.cc:460] Webserver started at http://127.3.82.3:32987/ using document root <none> and password file <none>
I20260925 09:20:52.904909 12542 fs_manager.cc:359] Metadata directory not provided
I20260925 09:20:52.905071 12542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:20:52.911093 12542 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:20:52.914373 12558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:52.916045 12542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 09:20:52.916247 12542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "661c8d8d40a449fa868a65ba55a8b7cb"
format_stamp: "Formatted at 2026-09-25 09:20:50 on dist-test-slave-4wm5"
server_key: "3a16f6bba6a449bbd6bbe589efe5622a"
server_key_iv: "4c424be19b449c369d8048e31a19e2b1"
server_key_version: "encryptionkey@0"
I20260925 09:20:52.917559 12542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:20:52.954730 12542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:20:52.960021 12542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:20:52.960366 12542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:20:52.961477 12542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:20:52.962953 12542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:20:52.963043 12542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:52.963122 12542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:20:52.963176 12542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:20:52.999415 12542 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:44017
I20260925 09:20:52.999450 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:44017 every 8 connection(s)
I20260925 09:20:53.000921 12542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:20:53.002923 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12542
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:20:53.022492 12543 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:53.004358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:56535 (local address 127.3.82.62:36655)
0925 09:20:53.004498 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:53.004505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:53.006231 (+ 1726us) server_negotiation.cc:407] Connection header received
0925 09:20:53.008223 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:53.008229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:53.008341 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:53.008443 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:53.011707 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.013116 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.015002 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.015544 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.015704 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.022153 (+ 6449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:56535: BlockingRecv error: recv got EOF from 127.3.82.3:56535 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 09:20:53.022986 12409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:53.004127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.3.82.62:36655)
0925 09:20:53.004333 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:53.004339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:53.004943 (+ 604us) server_negotiation.cc:407] Connection header received
0925 09:20:53.005207 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:53.005212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:53.005283 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:53.005404 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:53.007304 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.008328 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.009779 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.010384 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.010572 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.014313 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:20:53.014322 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:20:53.014342 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:20:53.014391 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:20:53.018412 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:53.019487 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:53.019502 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:53.019644 (+ 142us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:53.019926 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:20:53.020430 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:20:53.020435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:20:53.020439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:20:53.021264 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:20:53.021635 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0925 09:20:53.022205 (+ 570us) server_negotiation.cc:299] Negotiation successful
0925 09:20:53.022633 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:20:53.023341 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:53.005090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:56535)
0925 09:20:53.006031 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:53.006080 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:53.007939 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:53.010122 (+ 2183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:53.010132 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:53.010752 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:53.011416 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.011436 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.013360 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.013365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.014837 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.014852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.015818 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.015828 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.016133 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.016862 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:53.016887 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:53.021875 (+ 4988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":398,"threads_started":1}
W20260925 09:20:53.024060 12671 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:36655 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.82.62:36655: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:53.033934 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:53.024825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:43125)
0925 09:20:53.025007 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:53.025026 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:53.025133 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:53.025575 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:53.025580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:53.026064 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:53.026225 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.026234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.027537 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.027542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.028451 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.028462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.029251 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.029255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.029398 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.030114 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:53.030135 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:53.033594 (+ 3459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:20:53.034292 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:53.024917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:43125 (local address 127.3.82.62:36655)
0925 09:20:53.025076 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:53.025080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:53.025097 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:53.025278 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:53.025282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:53.025367 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:53.025505 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:53.026375 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.027405 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.028655 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.029115 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.029218 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.033984 (+ 4766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:43125: BlockingRecv error: recv got EOF from 127.3.82.3:43125 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:53.044387 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:53.034767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:45439)
0925 09:20:53.034945 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:53.034964 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:53.035118 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:53.035502 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:53.035506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:53.035841 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:53.035992 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.036000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.037085 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.037091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.038243 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.038256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.039232 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.039237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.039415 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.040197 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:53.040262 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:53.044078 (+ 3816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 09:20:53.044683 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:53.034909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:45439 (local address 127.3.82.62:36655)
0925 09:20:53.035229 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:53.035235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:53.035251 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:20:53.035312 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:53.035315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:53.035372 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:53.035477 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:53.036135 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.036937 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.038385 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.039062 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.039219 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.044363 (+ 5144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:45439: BlockingRecv error: recv got EOF from 127.3.82.3:45439 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
W20260925 09:20:53.044755 12671 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:53.051239 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:53.041175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:59161)
0925 09:20:53.041706 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:53.041733 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:53.041918 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:53.042436 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:53.042441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:53.043069 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:53.043354 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.043367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.044896 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.044901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.045984 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.045996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.046780 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.046786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.046910 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.047579 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:53.047604 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:53.050847 (+ 3243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":278,"threads_started":1}
W20260925 09:20:53.051468 12543 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:53.041226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:59161 (local address 127.3.82.62:36655)
0925 09:20:53.041425 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:53.041434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:53.041877 (+ 443us) server_negotiation.cc:407] Connection header received
0925 09:20:53.042138 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:53.042147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:53.042277 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:53.042385 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:53.043501 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.044578 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.046131 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.046592 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.046702 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.051128 (+ 4426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:59161: BlockingRecv error: recv got EOF from 127.3.82.1:59161 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:53 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:53.552548 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:53.542811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:47843)
0925 09:20:53.543331 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:53.543347 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:53.543517 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:53.543936 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:53.543940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:53.544447 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:53.544670 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.544680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.546038 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.546042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.547103 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:53.547116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.548105 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.548110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:53.548252 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.548842 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:53.548859 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:53.552210 (+ 3351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":294,"threads_started":1}
W20260925 09:20:53.552695 12543 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:53.542866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:47843 (local address 127.3.82.62:36655)
0925 09:20:53.543081 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:53.543088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:53.543440 (+ 352us) server_negotiation.cc:407] Connection header received
0925 09:20:53.543718 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:53.543725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:53.543797 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:53.543904 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:53.544850 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.545880 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.547263 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:53.547901 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:53.548020 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:53.552420 (+ 4400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:47843: BlockingRecv error: recv got EOF from 127.3.82.2:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 09:20:54 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:54 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:54.056268 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:54.045509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:43251)
0925 09:20:54.046290 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:54.046309 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:54.046514 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:54.047162 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:54.047169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:54.047755 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:54.048016 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:54.048028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.049258 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.049264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:54.050428 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:54.050444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.051556 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.051562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:54.051766 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:54.052780 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:54.052805 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:54.056021 (+ 3216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":425,"threads_started":1}
W20260925 09:20:54.056541 12543 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:54.045722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:43251 (local address 127.3.82.62:36655)
0925 09:20:54.046078 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:54.046086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:54.046430 (+ 344us) server_negotiation.cc:407] Connection header received
0925 09:20:54.046768 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:54.046773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:54.046890 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:54.047044 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:54.048229 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.049041 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.050649 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.051292 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.051431 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:54.056277 (+ 4846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:43251: BlockingRecv error: recv got EOF from 127.3.82.3:43251 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Sep 25 09:20:54 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:54 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:54.062814 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:54.052309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:35691)
0925 09:20:54.053238 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:54.053261 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:54.053464 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:54.053905 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:54.053911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:54.054373 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:54.054594 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:54.054611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.055944 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.055952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:54.057426 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:54.057447 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.058362 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.058368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:54.058580 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:54.059231 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:54.059251 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:54.062512 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":307,"threads_started":1}
W20260925 09:20:54.063330 12679 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:54.052442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:35691 (local address 127.3.82.62:36655)
0925 09:20:54.053187 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:54.053193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:54.053391 (+ 198us) server_negotiation.cc:407] Connection header received
0925 09:20:54.053646 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:54.053652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:54.053717 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:54.053838 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:54.054760 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.055793 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.057611 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.058152 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.058333 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:54.062877 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:35691: BlockingRecv error: recv got EOF from 127.3.82.1:35691 (error 108)
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":461,"threads_started":1}
Sep 25 09:20:54 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:54 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:54.563397 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:54.553969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:46117)
0925 09:20:54.554512 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:54.554536 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:54.554672 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:54.555028 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:54.555032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:54.555430 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:54.555581 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:54.555589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.556748 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.556757 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:54.558195 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:54.558219 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.559219 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.559225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:54.559376 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:54.560185 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:54.560204 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:54.563154 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":316,"threads_started":1}
W20260925 09:20:54.563817 12679 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:54.554225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:46117 (local address 127.3.82.62:36655)
0925 09:20:54.554480 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:54.554487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:54.554628 (+ 141us) server_negotiation.cc:407] Connection header received
0925 09:20:54.554792 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:54.554796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:54.554864 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:54.554951 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:54.555739 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.556493 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.558350 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:54.559027 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:54.559152 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:54.563498 (+ 4346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:46117: BlockingRecv error: recv got EOF from 127.3.82.2:46117 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 25 09:20:55 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:55 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:55.067973 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:55.057366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:37777)
0925 09:20:55.058005 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:55.058035 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:55.058293 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:55.058747 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:55.058752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:55.059290 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:55.059463 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:55.059473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.060693 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.060700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:55.061741 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:55.061754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.062657 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.062664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:55.062859 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:55.063580 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:55.063609 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:55.067400 (+ 3791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":294,"threads_started":1}
W20260925 09:20:55.067996 12679 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:55.057665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:37777 (local address 127.3.82.62:36655)
0925 09:20:55.057973 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:55.057979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:55.058130 (+ 151us) server_negotiation.cc:407] Connection header received
0925 09:20:55.058453 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:55.058459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:55.058559 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:55.058702 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:55.059637 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.060514 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.061973 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.062474 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.062635 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:55.067652 (+ 5017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:37777: BlockingRecv error: recv got EOF from 127.3.82.3:37777 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Sep 25 09:20:55 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:55 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:55.074589 12682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:55.064012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:60731)
0925 09:20:55.064776 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:55.064799 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:55.064949 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:55.065509 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:55.065513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:55.065961 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:55.066139 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:55.066148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.067720 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.067726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:55.069157 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:55.069171 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.070050 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.070055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:55.070182 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:55.070852 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:55.070873 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:55.074285 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":469,"threads_started":1}
W20260925 09:20:55.074837 12683 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:55.064194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:60731 (local address 127.3.82.62:36655)
0925 09:20:55.065137 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:55.065141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:55.065157 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:20:55.065202 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:55.065206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:55.065318 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:55.065447 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:55.066392 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.067540 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.069339 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.069913 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.070015 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:55.074555 (+ 4540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:60731: BlockingRecv error: recv got EOF from 127.3.82.1:60731 (error 108)
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":706,"threads_started":1}
Sep 25 09:20:55 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:55 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:55.582564 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:55.564814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:50221)
0925 09:20:55.565334 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:55.565363 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:55.565487 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:55.565910 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:55.565917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:55.566386 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:55.566568 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:55.566577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.567926 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.567931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:55.569075 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:55.569090 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.570002 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.570007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:55.570160 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:55.578378 (+ 8218us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:55.578404 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:55.582160 (+ 3756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":302,"threads_started":1}
W20260925 09:20:55.582683 12683 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:55.564913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:50221 (local address 127.3.82.62:36655)
0925 09:20:55.565138 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:55.565143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:55.565463 (+ 320us) server_negotiation.cc:407] Connection header received
0925 09:20:55.565612 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:55.565621 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:55.565716 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:55.565810 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:55.566831 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.567759 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.569264 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:55.569735 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:55.569957 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:55.582396 (+ 12439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:50221: BlockingRecv error: recv got EOF from 127.3.82.2:50221 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 09:20:56 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:56 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:56.078437 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:56.069278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:56601)
0925 09:20:56.069793 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:56.069809 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:56.069942 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:56.070271 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:56.070274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:56.070612 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:56.070861 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:56.070870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.071961 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.071965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:56.072907 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:56.072919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.073576 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.073580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:56.073715 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:56.074533 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:56.074552 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:56.078036 (+ 3484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":285,"threads_started":1}
W20260925 09:20:56.078562 12683 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:56.069443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:56601 (local address 127.3.82.62:36655)
0925 09:20:56.069615 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:56.069620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:56.069906 (+ 286us) server_negotiation.cc:407] Connection header received
0925 09:20:56.070043 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:56.070047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:56.070122 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:56.070207 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:56.071001 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.071804 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.073045 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.073432 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.073529 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:56.078254 (+ 4725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:56601: BlockingRecv error: recv got EOF from 127.3.82.3:56601 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 09:20:56 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:56 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:56.085670 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:56.075830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:37631)
0925 09:20:56.076569 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:56.076589 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:56.076749 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:56.077491 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:56.077497 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:56.077987 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:56.078199 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:56.078211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.079571 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.079578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:56.080841 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:56.080856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.081639 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.081645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:56.081871 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:56.082487 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:56.082505 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:56.085434 (+ 2929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":475,"threads_started":1}
W20260925 09:20:56.085999 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:56.075890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:37631 (local address 127.3.82.62:36655)
0925 09:20:56.077091 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:56.077096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:56.077113 (+ 17us) server_negotiation.cc:407] Connection header received
0925 09:20:56.077216 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:56.077225 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:56.077318 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:56.077418 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:56.078371 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.079411 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.081000 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.081487 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.081623 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:56.085660 (+ 4037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:37631: BlockingRecv error: recv got EOF from 127.3.82.1:37631 (error 108)
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":957,"threads_started":1}
Sep 25 09:20:56 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:56 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:56.595557 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:56.583819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:53541)
0925 09:20:56.586224 (+ 2405us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:56.586246 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:56.586457 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:56.586948 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:56.586953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:56.587376 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:56.587582 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:56.587598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.588773 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.588782 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:56.589762 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:56.589775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.590602 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.590608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:56.590832 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:56.591541 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:56.591559 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:56.595194 (+ 3635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1966,"thread_start_us":1829,"threads_started":1}
W20260925 09:20:56.595733 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:56.584664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:53541 (local address 127.3.82.62:36655)
0925 09:20:56.584974 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:56.584984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:56.586653 (+ 1669us) server_negotiation.cc:407] Connection header received
0925 09:20:56.586717 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:56.586721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:56.586788 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:56.586915 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:56.587754 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.588606 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.589972 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:56.590442 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:56.590590 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:56.595476 (+ 4886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:53541: BlockingRecv error: recv got EOF from 127.3.82.2:53541 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
Sep 25 09:20:57 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:57 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:57.091637 12689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:57.079781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:35187)
0925 09:20:57.080320 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:57.080338 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:57.080589 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:57.081337 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:57.081341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:57.081928 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:57.082162 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:57.082174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.084086 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.084093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:57.085763 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:57.085781 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.086876 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.086884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:57.087122 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:57.087872 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:57.087900 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:57.091403 (+ 3503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":510,"threads_started":1}
W20260925 09:20:57.092242 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:57.080042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:35187 (local address 127.3.82.62:36655)
0925 09:20:57.080239 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:57.080245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:57.080461 (+ 216us) server_negotiation.cc:407] Connection header received
0925 09:20:57.080824 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:57.080832 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:57.080962 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:57.081078 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:57.082351 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.083911 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.085956 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.086670 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.086823 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:57.091817 (+ 4994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:35187: BlockingRecv error: recv got EOF from 127.3.82.3:35187 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 09:20:57 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:57 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:57.102583 12690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:57.086722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:46647)
0925 09:20:57.087239 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:57.087258 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:57.087377 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:57.093074 (+ 5697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:57.093078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:57.093510 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:57.093693 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:57.093704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.095622 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.095626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:57.097034 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:57.097050 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.098442 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.098447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:57.098628 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:57.099237 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:57.099257 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:57.102337 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":473,"threads_started":1}
W20260925 09:20:57.102944 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:57.087078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:46647 (local address 127.3.82.62:36655)
0925 09:20:57.092573 (+ 5495us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:57.092583 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:57.092611 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:20:57.092692 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:57.092701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:57.092813 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:57.092962 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:57.093911 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.095415 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.097182 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.098067 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.098211 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:57.102577 (+ 4366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:46647: BlockingRecv error: recv got EOF from 127.3.82.1:46647 (error 108)
Metrics: {"server-negotiator.queue_time_us":5327,"thread_start_us":9408,"threads_started":1}
Sep 25 09:20:57 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:57 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
W20260925 09:20:57.606184 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:57.596983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:39089 (local address 127.3.82.62:36655)
0925 09:20:57.597210 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:57.597218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:57.597505 (+ 287us) server_negotiation.cc:407] Connection header received
0925 09:20:57.597722 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:57.597727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:57.597792 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:57.597955 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:57.598770 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.599498 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.600841 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.601260 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.601385 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:57.605781 (+ 4396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:39089: BlockingRecv error: recv got EOF from 127.3.82.2:39089 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:20:57.606264 12692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:57.596670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:39089)
0925 09:20:57.597390 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:57.597406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:57.597570 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:57.597974 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:57.597979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:57.598434 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:57.598597 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:57.598606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.599639 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.599647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:57.600696 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:57.600708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:57.601411 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:57.601416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:57.601584 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:57.602316 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:57.602333 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:57.605543 (+ 3210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":437,"threads_started":1}
Sep 25 09:20:58 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:58 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:58.101778 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:58.092803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:54109)
0925 09:20:58.093628 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:58.093646 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:58.093810 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:58.094214 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:58.094218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:58.094643 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:58.094821 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:58.094831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.095891 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.095898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:58.096830 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:58.096841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.097576 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.097593 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:58.097792 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:58.098488 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:58.098506 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:58.101416 (+ 2910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":535,"threads_started":1}
W20260925 09:20:58.101931 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:58.092885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:54109 (local address 127.3.82.62:36655)
0925 09:20:58.093125 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:58.093130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:58.093742 (+ 612us) server_negotiation.cc:407] Connection header received
0925 09:20:58.093975 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:58.093981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:58.094066 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:58.094177 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:58.094983 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.095743 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.096992 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.097413 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.097548 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:58.101653 (+ 4105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:54109: BlockingRecv error: recv got EOF from 127.3.82.3:54109 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 25 09:20:58 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:58 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:58.113086 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:58.103532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:48371)
0925 09:20:58.104107 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:58.104129 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:58.104271 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:58.104637 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:58.104642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:58.105126 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:58.105370 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:58.105383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.106907 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.106916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:58.108181 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:58.108195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.108971 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.108975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:58.109105 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:58.109715 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:58.109740 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:58.112847 (+ 3107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":320,"threads_started":1}
W20260925 09:20:58.113453 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:58.103588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48371 (local address 127.3.82.62:36655)
0925 09:20:58.103756 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:58.103762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:58.104219 (+ 457us) server_negotiation.cc:407] Connection header received
0925 09:20:58.104413 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:58.104417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:58.104479 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:58.104559 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:58.105527 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.106690 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.108334 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.108819 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.108916 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:58.113058 (+ 4142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:48371: BlockingRecv error: recv got EOF from 127.3.82.1:48371 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 25 09:20:58 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:58 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:58.615599 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:58.607208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:53703)
0925 09:20:58.607712 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:58.607727 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:58.607945 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:58.608288 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:58.608293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:58.608722 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:58.608885 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:58.608894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.609988 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.609993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:58.610918 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:58.610929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.611661 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.611668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:58.611850 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:58.612480 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:58.612498 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:58.615344 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":274,"threads_started":1}
W20260925 09:20:58.615909 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:58.607285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:53703 (local address 127.3.82.62:36655)
0925 09:20:58.607531 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:58.607538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:58.607865 (+ 327us) server_negotiation.cc:407] Connection header received
0925 09:20:58.608087 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:58.608097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:58.608157 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:58.608257 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:58.609032 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.609786 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.611077 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:58.611501 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:58.611642 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:58.615660 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:53703: BlockingRecv error: recv got EOF from 127.3.82.2:53703 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 09:20:59 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:59 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:59.112429 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:59.102820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:37305)
0925 09:20:59.103354 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:59.103371 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:59.103487 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:59.103814 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:59.103817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:59.104403 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:59.104556 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:59.104565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.105649 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.105654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:59.106642 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:59.106654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.107566 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.107573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:59.107751 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:59.108478 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:59.108499 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:59.112211 (+ 3712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":291,"threads_started":1}
W20260925 09:20:59.112828 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:59.102994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:37305 (local address 127.3.82.62:36655)
0925 09:20:59.103231 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:59.103237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:59.103471 (+ 234us) server_negotiation.cc:407] Connection header received
0925 09:20:59.103593 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:59.103597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:59.103677 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:59.103850 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:59.104710 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.105498 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.106813 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.107327 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.107447 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:59.112467 (+ 5020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:37305: BlockingRecv error: recv got EOF from 127.3.82.3:37305 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 25 09:20:59 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:59 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:59.123840 12697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:59.114239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:35613)
0925 09:20:59.114789 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:59.114806 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:59.114967 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:59.115343 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:59.115348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:59.115786 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:59.116024 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:59.116034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.117074 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.117080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:59.118192 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:59.118210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.119036 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.119042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:59.119218 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:59.119853 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:59.119872 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:59.123595 (+ 3723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":244,"threads_started":1}
W20260925 09:20:59.124233 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:59.114328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:35613 (local address 127.3.82.62:36655)
0925 09:20:59.114530 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:59.114536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:59.114894 (+ 358us) server_negotiation.cc:407] Connection header received
0925 09:20:59.115109 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:59.115114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:59.115185 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:59.115299 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:59.116163 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.116907 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.118399 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.118870 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.119027 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:59.123815 (+ 4788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:35613: BlockingRecv error: recv got EOF from 127.3.82.1:35613 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 09:20:59 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:20:59 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:20:59.626813 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:20:59.616847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:53483)
0925 09:20:59.617515 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0925 09:20:59.617532 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:20:59.617657 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:20:59.618213 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:20:59.618217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:20:59.618603 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:20:59.618784 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:59.618792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.620024 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.620029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:59.621333 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:20:59.621348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.622555 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.622563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:20:59.622758 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:59.623430 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:20:59.623449 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:20:59.626445 (+ 2996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":320,"threads_started":1}
W20260925 09:20:59.626945 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:20:59.616894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:53483 (local address 127.3.82.62:36655)
0925 09:20:59.617169 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 09:20:59.617176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:20:59.617623 (+ 447us) server_negotiation.cc:407] Connection header received
0925 09:20:59.617901 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:20:59.617910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:20:59.617982 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:20:59.618043 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:20:59.619040 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.619844 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.621514 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:20:59.622377 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:20:59.622502 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:20:59.626673 (+ 4171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:53483: BlockingRecv error: recv got EOF from 127.3.82.2:53483 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Sep 25 09:21:00 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:00 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:00.123272 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:00.113450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:36593)
0925 09:21:00.114186 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:00.114203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:00.114333 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:00.114652 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:00.114655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:00.115091 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:00.115307 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:00.115318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.116496 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.116502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:00.117615 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:00.117628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.118589 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.118594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:00.118716 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:00.119347 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:00.119365 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:00.122912 (+ 3547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":415,"threads_started":1}
W20260925 09:21:00.123469 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:00.113569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:36593 (local address 127.3.82.62:36655)
0925 09:21:00.113818 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:00.113852 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:00.114301 (+ 449us) server_negotiation.cc:407] Connection header received
0925 09:21:00.114437 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:00.114441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:00.114520 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:00.114620 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:00.115471 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.116297 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.117767 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.118381 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.118512 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:00.123168 (+ 4656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:36593: BlockingRecv error: recv got EOF from 127.3.82.3:36593 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 25 09:21:00 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:00 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:00.134501 12700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:00.124824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:48049)
0925 09:21:00.125383 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:00.125404 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:00.125552 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:00.125949 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:00.125954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:00.126439 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:00.126591 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:00.126600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.127759 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.127768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:00.128722 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:00.128733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.129507 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.129513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:00.129701 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:00.130454 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:00.130478 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:00.134106 (+ 3628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":318,"threads_started":1}
W20260925 09:21:00.134768 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:00.125094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48049 (local address 127.3.82.62:36655)
0925 09:21:00.125301 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:00.125307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:00.125486 (+ 179us) server_negotiation.cc:407] Connection header received
0925 09:21:00.125702 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:00.125708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:00.125789 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:00.125927 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:00.126833 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.127608 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.128883 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.129307 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.129444 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:00.134360 (+ 4916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:48049: BlockingRecv error: recv got EOF from 127.3.82.1:48049 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 25 09:21:00 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:00 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:00.644982 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:00.627875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:52273)
0925 09:21:00.628475 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:00.628492 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:00.628692 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:00.629116 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:00.629121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:00.629712 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:00.629932 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:00.629941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.631189 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.631194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:00.632309 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:00.632322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.633144 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.633149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:00.633332 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:00.641287 (+ 7955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:00.641319 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:00.644739 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":364,"threads_started":1}
W20260925 09:21:00.645341 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:00.628124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:52273 (local address 127.3.82.62:36655)
0925 09:21:00.628349 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:00.628355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:00.628580 (+ 225us) server_negotiation.cc:407] Connection header received
0925 09:21:00.628845 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:00.628850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:00.628941 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:00.629035 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:00.630079 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.631031 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.632452 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:00.632941 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:00.633036 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:00.645073 (+ 12037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:52273: BlockingRecv error: recv got EOF from 127.3.82.2:52273 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 09:21:01 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:01 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:01.138104 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:01.124549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:41865)
0925 09:21:01.125132 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:01.125149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:01.125338 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:01.125814 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:01.125853 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:01.126589 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:01.126826 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:01.126842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.131234 (+ 4392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.131241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:01.132344 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:01.132359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.133212 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.133218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:01.133399 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:01.134136 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:01.134155 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:01.137709 (+ 3554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":321,"threads_started":1}
W20260925 09:21:01.138387 12687 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:01.124663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:41865 (local address 127.3.82.62:36655)
0925 09:21:01.124971 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:01.124977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:01.125254 (+ 277us) server_negotiation.cc:407] Connection header received
0925 09:21:01.125493 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:01.125501 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:01.125636 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:01.125752 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:01.130047 (+ 4295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.131045 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.132513 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.132989 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.133159 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:01.138027 (+ 4868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:41865: BlockingRecv error: recv got EOF from 127.3.82.3:41865 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Sep 25 09:21:01 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:01 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:01.148738 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:01.135781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:45089)
0925 09:21:01.136376 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:01.136393 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:01.136540 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:01.137217 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:01.137223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:01.137706 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:01.137962 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:01.137975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.139393 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.139398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:01.141468 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:01.141489 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.142887 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.142927 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:01.143205 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:01.144227 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:01.144262 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:01.148470 (+ 4208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":318,"threads_started":1}
W20260925 09:21:01.149242 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:01.135875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:45089 (local address 127.3.82.62:36655)
0925 09:21:01.136860 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:01.136864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:01.136880 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:21:01.136963 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:01.136969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:01.137043 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:01.137135 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:01.138140 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.139212 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.141750 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.142596 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.142764 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:01.148803 (+ 6039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:45089: BlockingRecv error: recv got EOF from 127.3.82.1:45089 (error 108)
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":733,"threads_started":1}
Sep 25 09:21:01 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:01 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:01.656144 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:01.646108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:43197)
0925 09:21:01.646842 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:01.646861 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:01.646992 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:01.647426 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:01.647430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:01.647905 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:01.648101 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:01.648112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.649320 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.649329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:01.650443 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:01.650458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.651292 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.651297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:01.651440 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:01.652206 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:01.652227 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:01.655803 (+ 3576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":352,"threads_started":1}
W20260925 09:21:01.656387 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:01.646168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:43197 (local address 127.3.82.62:36655)
0925 09:21:01.646393 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:01.646398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:01.646952 (+ 554us) server_negotiation.cc:407] Connection header received
0925 09:21:01.647110 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:01.647114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:01.647181 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:01.647273 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:01.648299 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.649126 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.650619 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:01.651086 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:01.651202 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:01.656136 (+ 4934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:43197: BlockingRecv error: recv got EOF from 127.3.82.2:43197 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 25 09:21:02 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:02 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:02.149389 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:02.139417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:42205)
0925 09:21:02.140176 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:02.140199 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:02.140340 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:02.140802 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:02.140806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:02.141197 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:02.141451 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:02.141464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.142867 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.142871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:02.143897 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:02.143909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.144783 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.144788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:02.144908 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:02.145518 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:02.145537 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:02.149030 (+ 3493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":435,"threads_started":1}
W20260925 09:21:02.149761 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:02.139485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:42205 (local address 127.3.82.62:36655)
0925 09:21:02.139737 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:02.139745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:02.140284 (+ 539us) server_negotiation.cc:407] Connection header received
0925 09:21:02.140514 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:02.140519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:02.140599 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:02.140688 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:02.141596 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.142708 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.144045 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.144540 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.144727 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:02.149298 (+ 4571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:42205: BlockingRecv error: recv got EOF from 127.3.82.3:42205 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 25 09:21:02 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:02 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:02.159370 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:02.149887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:48179)
0925 09:21:02.150425 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:02.150445 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:02.150571 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:02.151144 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:02.151148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:02.151612 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:02.151759 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:02.151767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.152876 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.152883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:02.153899 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:02.153919 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.154665 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.154671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:02.154845 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:02.155501 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:02.155519 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:02.158968 (+ 3449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":298,"threads_started":1}
W20260925 09:21:02.159570 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:02.149944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48179 (local address 127.3.82.62:36655)
0925 09:21:02.150548 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:02.150553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:02.150572 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:21:02.150770 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:02.150777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:02.150890 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:02.150977 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:02.151913 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.152739 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.154052 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.154497 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.154638 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:02.159289 (+ 4651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:48179: BlockingRecv error: recv got EOF from 127.3.82.1:48179 (error 108)
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":777,"threads_started":1}
Sep 25 09:21:02 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:02 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:02.666980 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:02.657351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:46531)
0925 09:21:02.658011 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:02.658029 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:02.658208 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:02.658586 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:02.658592 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:02.659066 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:02.659243 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:02.659254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.660779 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.660785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:02.661932 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:02.661946 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.662841 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.662846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:02.662972 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:02.663641 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:02.663660 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:02.666742 (+ 3082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":428,"threads_started":1}
W20260925 09:21:02.667258 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:02.657542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:46531 (local address 127.3.82.62:36655)
0925 09:21:02.657790 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:02.657799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:02.658122 (+ 323us) server_negotiation.cc:407] Connection header received
0925 09:21:02.658370 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:02.658380 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:02.658446 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:02.658558 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:02.659568 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.660660 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.662135 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:02.662681 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:02.662800 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:02.667000 (+ 4200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:46531: BlockingRecv error: recv got EOF from 127.3.82.2:46531 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 25 09:21:03 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:03 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:03.161463 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:03.151374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:57337)
0925 09:21:03.152076 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:03.152099 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:03.152311 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:03.152738 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:03.152743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:03.153252 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:03.153467 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:03.153480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.154777 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.154781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:03.155868 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:03.155881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.156657 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.156662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:03.156791 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:03.157537 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:03.157556 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:03.161066 (+ 3510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":461,"threads_started":1}
W20260925 09:21:03.161581 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:03.151296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:57337 (local address 127.3.82.62:36655)
0925 09:21:03.151673 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:03.151680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:03.152193 (+ 513us) server_negotiation.cc:407] Connection header received
0925 09:21:03.152467 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:03.152473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:03.152569 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:03.152685 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:03.153738 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.154609 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.156046 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.156495 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.156606 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:03.161320 (+ 4714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:57337: BlockingRecv error: recv got EOF from 127.3.82.3:57337 (error 108)
Metrics: {"server-negotiator.queue_time_us":221}
Sep 25 09:21:03 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:03 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:03.171931 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:03.160599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:54245)
0925 09:21:03.162932 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:03.162957 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:03.163143 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:03.163615 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:03.163620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:03.164172 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:03.164469 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:03.164486 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.165526 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.165532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:03.166594 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:03.166607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.167416 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.167422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:03.167599 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:03.168402 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:03.168421 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:03.171547 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2212,"thread_start_us":2087,"threads_started":1}
W20260925 09:21:03.172111 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:03.160788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:54245 (local address 127.3.82.62:36655)
0925 09:21:03.161864 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:03.161872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:03.163081 (+ 1209us) server_negotiation.cc:407] Connection header received
0925 09:21:03.163346 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:03.163352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:03.163437 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:03.163570 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:03.164627 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.165376 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.166757 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.167232 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.167397 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:03.171800 (+ 4403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:54245: BlockingRecv error: recv got EOF from 127.3.82.1:54245 (error 108)
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":1542,"threads_started":1}
Sep 25 09:21:03 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:03 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:03.677351 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:03.668204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:41151)
0925 09:21:03.668824 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:03.668843 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:03.668966 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:03.669410 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:03.669415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:03.669879 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:03.670060 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:03.670069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.671175 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.671179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:03.672143 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:03.672157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.673177 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.673182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:03.673339 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:03.674099 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:03.674118 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:03.677092 (+ 2974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":270,"threads_started":1}
W20260925 09:21:03.677676 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:03.668346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41151 (local address 127.3.82.62:36655)
0925 09:21:03.668567 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:03.668573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:03.668954 (+ 381us) server_negotiation.cc:407] Connection header received
0925 09:21:03.669097 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:03.669103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:03.669234 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:03.669364 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:03.670274 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.671036 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.672374 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:03.672999 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:03.673120 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:03.677388 (+ 4268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:41151: BlockingRecv error: recv got EOF from 127.3.82.2:41151 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 25 09:21:04 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:04 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:04.172783 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:04.162943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:57289)
0925 09:21:04.163504 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:04.163521 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:04.163637 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:04.164005 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:04.164008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:04.164450 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:04.164641 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:04.164649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.165739 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.165743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:04.166766 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:04.166778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.167793 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.167797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:04.167928 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:04.168647 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:04.168673 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:04.172391 (+ 3718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":305,"threads_started":1}
W20260925 09:21:04.173151 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:04.163037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:57289 (local address 127.3.82.62:36655)
0925 09:21:04.163256 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:04.163262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:04.163605 (+ 343us) server_negotiation.cc:407] Connection header received
0925 09:21:04.163793 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:04.163798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:04.163865 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:04.163954 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:04.164820 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.165583 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.166992 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.167556 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.167688 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:04.172798 (+ 5110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:57289: BlockingRecv error: recv got EOF from 127.3.82.3:57289 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 25 09:21:04 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:04 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:04.183020 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:04.173053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:34373)
0925 09:21:04.173741 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:04.173757 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:04.173966 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:04.174387 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:04.174392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:04.174815 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:04.174983 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:04.174992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.176100 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.176107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:04.177226 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:04.177263 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.178225 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.178233 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:04.178455 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:04.179163 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:04.179183 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:04.182626 (+ 3443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":399,"threads_started":1}
W20260925 09:21:04.183126 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:04.173286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:34373 (local address 127.3.82.62:36655)
0925 09:21:04.173608 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:04.173615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:04.173879 (+ 264us) server_negotiation.cc:407] Connection header received
0925 09:21:04.174134 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:04.174139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:04.174204 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:04.174309 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:04.175150 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.175944 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.177432 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.178032 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.178201 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:04.182866 (+ 4665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:34373: BlockingRecv error: recv got EOF from 127.3.82.1:34373 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":522,"threads_started":1}
Sep 25 09:21:04 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:04 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:04.688369 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:04.678531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:50081)
0925 09:21:04.679062 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:04.679080 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:04.679215 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:04.679555 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:04.679558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:04.679971 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:04.680131 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:04.680140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.681695 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.681702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:04.683185 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:04.683199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.684223 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.684227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:04.684371 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:04.685067 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:04.685086 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:04.688130 (+ 3044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":295,"threads_started":1}
W20260925 09:21:04.688894 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:04.678650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:50081 (local address 127.3.82.62:36655)
0925 09:21:04.679016 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:04.679022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:04.679219 (+ 197us) server_negotiation.cc:407] Connection header received
0925 09:21:04.679319 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:04.679324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:04.679404 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:04.679508 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:04.680302 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.681515 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.683365 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:04.684042 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:04.684179 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:04.688411 (+ 4232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:50081: BlockingRecv error: recv got EOF from 127.3.82.2:50081 (error 108)
Metrics: {"server-negotiator.queue_time_us":218}
Sep 25 09:21:05 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:05 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:05.183154 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:05.174157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:44299)
0925 09:21:05.174773 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:05.174789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:05.174904 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:05.175307 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:05.175310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:05.175708 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:05.175877 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:05.175886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.176992 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.176996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:05.177965 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:05.177977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.178694 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.178698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:05.178850 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:05.179628 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:05.179645 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:05.182855 (+ 3210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":357,"threads_started":1}
W20260925 09:21:05.183425 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:05.174267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:44299 (local address 127.3.82.62:36655)
0925 09:21:05.174481 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:05.174486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:05.174894 (+ 408us) server_negotiation.cc:407] Connection header received
0925 09:21:05.175003 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:05.175007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:05.175080 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:05.175205 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:05.176043 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.176843 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.178102 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.178556 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.178669 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:05.183157 (+ 4488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:44299: BlockingRecv error: recv got EOF from 127.3.82.3:44299 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 25 09:21:05 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:05 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:05.193006 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:05.184151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:45537)
0925 09:21:05.184641 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:05.184660 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:05.184784 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:05.185127 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:05.185131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:05.185506 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:05.185690 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:05.185701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.186833 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.186838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:05.187972 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:05.187987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.188800 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.188806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:05.188948 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:05.189567 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:05.189588 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:05.192770 (+ 3182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":252,"threads_started":1}
W20260925 09:21:05.193362 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:05.184256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:45537 (local address 127.3.82.62:36655)
0925 09:21:05.184481 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:05.184487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:05.184744 (+ 257us) server_negotiation.cc:407] Connection header received
0925 09:21:05.184898 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:05.184902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:05.184962 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:05.185042 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:05.185922 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.186668 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.188115 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.188546 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.188679 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:05.193077 (+ 4398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:45537: BlockingRecv error: recv got EOF from 127.3.82.1:45537 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 25 09:21:05 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:05 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:05.704896 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:05.689480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:35765)
0925 09:21:05.690311 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:05.690330 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:05.690457 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:05.690949 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:05.690954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:05.691431 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:05.691719 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:05.691733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.692813 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.692818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:05.693971 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:05.693986 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.694848 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.694854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:05.694987 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:05.701170 (+ 6183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:05.701193 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:05.704589 (+ 3396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":449,"threads_started":1}
W20260925 09:21:05.705112 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:05.689906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:35765 (local address 127.3.82.62:36655)
0925 09:21:05.690175 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:05.690180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:05.690420 (+ 240us) server_negotiation.cc:407] Connection header received
0925 09:21:05.690622 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:05.690626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:05.690759 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:05.690854 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:05.691882 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.692648 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.694149 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:05.694630 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:05.694740 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:05.704846 (+ 10106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:35765: BlockingRecv error: recv got EOF from 127.3.82.2:35765 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 25 09:21:06 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:06 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:06.193379 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:06.184244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:37009)
0925 09:21:06.185000 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:06.185022 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:06.185167 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:06.185626 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:06.185629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:06.186120 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:06.186311 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:06.186320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.187374 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.187378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:06.188384 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:06.188397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.189218 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.189222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:06.189347 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:06.190091 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:06.190109 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:06.193138 (+ 3029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":484,"threads_started":1}
W20260925 09:21:06.193775 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:06.184303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:37009 (local address 127.3.82.62:36655)
0925 09:21:06.184537 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:06.184557 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:06.185111 (+ 554us) server_negotiation.cc:407] Connection header received
0925 09:21:06.185303 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:06.185307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:06.185372 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:06.185453 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:06.186478 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.187228 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.188571 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.189044 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.189183 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:06.193490 (+ 4307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:37009: BlockingRecv error: recv got EOF from 127.3.82.3:37009 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 25 09:21:06 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:06 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:06.203482 12722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:06.194198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:59929)
0925 09:21:06.194818 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:06.194836 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:06.195004 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:06.195394 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:06.195399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:06.195830 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:06.195986 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:06.195995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.197140 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.197146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:06.198245 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:06.198258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.198997 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.199004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:06.199189 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:06.199893 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:06.199912 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:06.203105 (+ 3193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":317,"threads_started":1}
W20260925 09:21:06.203595 12704 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:06.194253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:59929 (local address 127.3.82.62:36655)
0925 09:21:06.194570 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:06.194579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:06.194927 (+ 348us) server_negotiation.cc:407] Connection header received
0925 09:21:06.195149 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:06.195156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:06.195236 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:06.195348 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:06.196137 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.196978 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.198392 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.198834 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.198970 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:06.203352 (+ 4382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:59929: BlockingRecv error: recv got EOF from 127.3.82.1:59929 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
Sep 25 09:21:06 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:06 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:06.715679 12723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:06.706020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:33147)
0925 09:21:06.706588 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:06.706885 (+ 297us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:06.707048 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:06.707543 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:06.707548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:06.707986 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:06.708262 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:06.708274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.709567 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.709571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:06.710608 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:06.710621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.711356 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.711361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:06.711528 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:06.712429 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:06.712448 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:06.715421 (+ 2973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":240,"threads_started":1}
W20260925 09:21:06.715965 12724 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:06.706240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:33147 (local address 127.3.82.62:36655)
0925 09:21:06.706864 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:06.706868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:06.706952 (+ 84us) server_negotiation.cc:407] Connection header received
0925 09:21:06.707239 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:06.707247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:06.707345 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:06.707478 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:06.708415 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.709422 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.710749 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:06.711186 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:06.711295 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:06.715655 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:33147: BlockingRecv error: recv got EOF from 127.3.82.2:33147 (error 108)
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":429,"threads_started":1}
Sep 25 09:21:07 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:07 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.3@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:07.204322 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:07.194600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.3:39465)
0925 09:21:07.195400 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:07.195419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:07.195597 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:07.195982 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:07.195987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:07.196461 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:07.196629 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:07.196639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.197761 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.197775 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:07.198854 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:07.198866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.199724 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.199730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:07.199921 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:07.200608 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:07.200628 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:07.203912 (+ 3284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":585,"threads_started":1}
W20260925 09:21:07.204408 12724 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:07.194731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:39465 (local address 127.3.82.62:36655)
0925 09:21:07.194956 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:07.194965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:07.195524 (+ 559us) server_negotiation.cc:407] Connection header received
0925 09:21:07.195736 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:07.195742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:07.195823 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:07.195945 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:07.196789 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.197587 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.199019 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.199517 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.199693 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:07.204180 (+ 4487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.3:39465: BlockingRecv error: recv got EOF from 127.3.82.3:39465 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 09:21:07 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:07 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.1@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:07.215117 12726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:07.204993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.1:36633)
0925 09:21:07.205599 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:07.205615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:07.205815 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:07.206315 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:07.206320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:07.207001 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:07.207260 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:07.207271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.208706 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.208710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:07.210060 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:07.210074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.210835 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.210840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:07.210965 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:07.211727 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:07.211748 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:07.214848 (+ 3100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":345,"threads_started":1}
W20260925 09:21:07.215252 12724 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:07.205083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:36633 (local address 127.3.82.62:36655)
0925 09:21:07.205365 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:07.205373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:07.205728 (+ 355us) server_negotiation.cc:407] Connection header received
0925 09:21:07.206046 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:07.206057 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:07.206126 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:07.206278 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:07.207413 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.208520 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.210198 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.210675 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.210781 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:07.214995 (+ 4214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.1:36633: BlockingRecv error: recv got EOF from 127.3.82.1:36633 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Sep 25 09:21:07 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
Sep 25 09:21:07 dist-test-slave-4wm5 krb5kdc[11671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.82.2@KRBTEST.COM for oryx/127.3.82.62@KRBTEST.COM, Server not found in Kerberos database
I20260925 09:21:07.727088 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:07.717164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36655 (local address 127.3.82.2:33247)
0925 09:21:07.717684 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:07.717702 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:07.718026 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:07.718477 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:07.718528 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:07.719059 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:07.719269 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:07.719280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.720460 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.720465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:07.721623 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:07.721635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.722604 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.722611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:07.722793 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:07.723464 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:07.723483 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:07.726828 (+ 3345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.82.62:36655: Server oryx/127.3.82.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":292,"threads_started":1}
W20260925 09:21:07.727487 12728 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:07.717216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:33247 (local address 127.3.82.62:36655)
0925 09:21:07.717749 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:07.717753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:07.717773 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:21:07.718181 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:07.718185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:07.718251 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:07.718375 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:07.719448 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.720320 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.721783 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:07.722449 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:07.722582 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:07.727169 (+ 4587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.82.2:33247: BlockingRecv error: recv got EOF from 127.3.82.2:33247 (error 108)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":320,"threads_started":1}
I20260925 09:21:08.003902 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12260
I20260925 09:21:08.019372 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:08.005470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.3.82.1:34077)
0925 09:21:08.006145 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:08.006153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:08.006181 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:21:08.006252 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:08.006259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:08.006510 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:08.006679 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:08.008280 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.009436 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.011126 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.012029 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.012156 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:08.015127 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:08.015137 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:08.015153 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:08.015202 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:08.018792 (+ 3590us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.82.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 09:21:08.019113 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49340: Request ticket server kudu/127.3.82.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":356,"threads_started":1}
W20260925 09:21:08.019507 12730 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49340: Request ticket server kudu/127.3.82.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 09:21:08.019624 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12260: Not authorized: Client connection negotiation failed: client connection to 127.3.82.1:34077: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.82.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 09:21:08.019913 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12260
I20260925 09:21:08.039592 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12411
I20260925 09:21:08.054818 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:08.041074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.3.82.2:36239)
0925 09:21:08.041956 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:08.041974 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:08.042008 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:21:08.042085 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:08.042117 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:08.042445 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:08.042629 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:08.044241 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.045378 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.047078 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.047751 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.047964 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:08.051173 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:08.051184 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:08.051200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:08.051249 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:08.054238 (+ 2989us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.82.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 09:21:08.054563 (+ 325us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37520: Request ticket server kudu/127.3.82.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":348,"threads_started":1}
W20260925 09:21:08.054965 12731 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37520: Request ticket server kudu/127.3.82.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 09:21:08.055135 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12411: Not authorized: Client connection negotiation failed: client connection to 127.3.82.2:36239: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.82.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 09:21:08.055357 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12411
I20260925 09:21:08.073319 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12542
I20260925 09:21:08.087234 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:08.074445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.3.82.3:44017)
0925 09:21:08.075150 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:08.075159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:08.075189 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:21:08.075257 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:08.075264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:08.075533 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:08.075716 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:08.077127 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.078462 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.079978 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.080594 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.080715 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:08.083868 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:08.083881 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:08.083899 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:08.083945 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:08.086680 (+ 2735us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.82.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 09:21:08.086983 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50900: Request ticket server kudu/127.3.82.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":402,"threads_started":1}
W20260925 09:21:08.087322 12732 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50900: Request ticket server kudu/127.3.82.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 09:21:08.087591 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12542: Not authorized: Client connection negotiation failed: client connection to 127.3.82.3:44017: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.82.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 09:21:08.087776 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12542
I20260925 09:21:08.104172 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12191
I20260925 09:21:08.105716 12253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:08.226006 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12191
2026-09-25T09:21:08Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21525 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T09:21:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:21:08Z Disabled control of system clock
I20260925 09:21:08.307168 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36029
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 09:21:08.662889 12737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:08.663218 12737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:08.663298 12737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:08.670864 12737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:21:08.670990 12737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:08.671034 12737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:08.671085 12737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:21:08.671131 12737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:21:08.680601 12737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:08.681950 12737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:08.683687 12737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:08.697520 12742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:08.697544 12743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:08.697467 12745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:08.697948 12737 server_base.cc:1034] running on GCE node
I20260925 09:21:08.699086 12737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:08.700766 12737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:08.702024 12737 hybrid_clock.cc:648] HybridClock initialized: now 1790328068701969 us; error 73 us; skew 500 ppm
I20260925 09:21:08.702451 12737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:08.705066 12737 webserver.cc:460] Webserver started at http://127.3.82.62:39013/ using document root <none> and password file <none>
I20260925 09:21:08.705931 12737 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:08.706075 12737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:08.706471 12737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:08.709798 12737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "1c3eb747c02346c5aabaa6fe5a7e1029"
format_stamp: "Formatted at 2026-09-25 09:21:08 on dist-test-slave-4wm5"
server_key: "bd5e9c38b535ad440a6961043f85a4e4"
server_key_iv: "a59eeaa32fb06e0b856ac576309e0743"
server_key_version: "encryptionkey@0"
I20260925 09:21:08.710830 12737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "1c3eb747c02346c5aabaa6fe5a7e1029"
format_stamp: "Formatted at 2026-09-25 09:21:08 on dist-test-slave-4wm5"
server_key: "bd5e9c38b535ad440a6961043f85a4e4"
server_key_iv: "a59eeaa32fb06e0b856ac576309e0743"
server_key_version: "encryptionkey@0"
I20260925 09:21:08.716799 12737 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:21:08.721297 12751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:08.722967 12737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:21:08.723186 12737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "1c3eb747c02346c5aabaa6fe5a7e1029"
format_stamp: "Formatted at 2026-09-25 09:21:08 on dist-test-slave-4wm5"
server_key: "bd5e9c38b535ad440a6961043f85a4e4"
server_key_iv: "a59eeaa32fb06e0b856ac576309e0743"
server_key_version: "encryptionkey@0"
I20260925 09:21:08.723443 12737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:08.761456 12737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:08.762581 12737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:08.762931 12737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:08.783293 12737 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36029
I20260925 09:21:08.783314 12802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36029 every 8 connection(s)
I20260925 09:21:08.784745 12737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:21:08.788910 12803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:21:08.792871 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12737
I20260925 09:21:08.793102 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:21:08.793896 3400 external_mini_cluster.cc:1428] Setting key 9774b6129f1f876e20434b2e15af8ece
I20260925 09:21:08.801252 12803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029: Bootstrap starting.
I20260925 09:21:08.805251 12803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029: Neither blocks nor log segments found. Creating new log.
I20260925 09:21:08.806700 12803 log.cc:824] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029: Log is configured to *not* fsync() on all Append() calls
I20260925 09:21:08.810746 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:08.796239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.3.82.62:36029)
0925 09:21:08.797356 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:08.797368 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:08.797421 (+ 53us) server_negotiation.cc:407] Connection header received
0925 09:21:08.799565 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:08.799594 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:08.800186 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:08.800580 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:08.801517 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.803093 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.805085 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:08.806015 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:08.806164 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:08.807859 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:08.807878 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:08.807905 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:08.808715 (+ 810us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:08.808802 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:08.808830 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:08.808918 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:08.809263 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 09:21:08.809671 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":418,"threads_started":1}
I20260925 09:21:08.810868 12803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029: No bootstrap required, opened a new log
I20260925 09:21:08.818480 12803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36029 } }
I20260925 09:21:08.818935 12803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:21:08.819078 12803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c3eb747c02346c5aabaa6fe5a7e1029, State: Initialized, Role: FOLLOWER
I20260925 09:21:08.819664 12803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36029 } }
I20260925 09:21:08.819893 12803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:21:08.819993 12803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:21:08.820133 12803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:21:08.822480 12803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36029 } }
I20260925 09:21:08.823035 12803 leader_election.cc:302] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c3eb747c02346c5aabaa6fe5a7e1029; no voters:
I20260925 09:21:08.823647 12803 leader_election.cc:288] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:21:08.823880 12808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:21:08.824769 12808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [term 1 LEADER]: Becoming Leader. State: Replica: 1c3eb747c02346c5aabaa6fe5a7e1029, State: Running, Role: LEADER
I20260925 09:21:08.825570 12808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36029 } }
I20260925 09:21:08.826217 12803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:21:08.828809 12810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c3eb747c02346c5aabaa6fe5a7e1029. Latest consensus state: current_term: 1 leader_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36029 } } }
I20260925 09:21:08.828843 12809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3eb747c02346c5aabaa6fe5a7e1029" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36029 } } }
I20260925 09:21:08.829414 12810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [sys.catalog]: This master's current role is: LEADER
I20260925 09:21:08.829670 12809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029 [sys.catalog]: This master's current role is: LEADER
I20260925 09:21:08.831883 12813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:21:08.838712 12813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:21:08.848249 12813 catalog_manager.cc:1347] Generated new cluster ID: bb55ea3181e046ddae61d59f90c6bb80
I20260925 09:21:08.848385 12813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:21:08.866444 12813 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:21:08.867877 12813 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:21:08.878312 12813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c3eb747c02346c5aabaa6fe5a7e1029: Generated new TSK 0
I20260925 09:21:08.879344 12813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:21:08.890232 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36029
--builtin_ntp_servers=127.3.82.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 09:21:09.250262 12827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:09.250563 12827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:09.250650 12827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:09.258312 12827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:09.258430 12827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:09.258540 12827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:21:09.268306 12827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:09.269595 12827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:09.271353 12827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:09.285374 12832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:09.285437 12833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:09.285382 12835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:09.286841 12827 server_base.cc:1034] running on GCE node
I20260925 09:21:09.287575 12827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:09.288697 12827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:09.289976 12827 hybrid_clock.cc:648] HybridClock initialized: now 1790328069289907 us; error 53 us; skew 500 ppm
I20260925 09:21:09.290373 12827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:09.293085 12827 webserver.cc:460] Webserver started at http://127.3.82.1:33229/ using document root <none> and password file <none>
I20260925 09:21:09.293939 12827 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:09.294077 12827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:09.294430 12827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:09.297480 12827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "eff90f98e7b94586b98067ca32d3788d"
format_stamp: "Formatted at 2026-09-25 09:21:09 on dist-test-slave-4wm5"
server_key: "8b71ee90ea378be8c9444d7c1a19c718"
server_key_iv: "78974b1ae3e4918138d5a4e53cb3aebc"
server_key_version: "encryptionkey@0"
I20260925 09:21:09.298408 12827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "eff90f98e7b94586b98067ca32d3788d"
format_stamp: "Formatted at 2026-09-25 09:21:09 on dist-test-slave-4wm5"
server_key: "8b71ee90ea378be8c9444d7c1a19c718"
server_key_iv: "78974b1ae3e4918138d5a4e53cb3aebc"
server_key_version: "encryptionkey@0"
I20260925 09:21:09.303964 12827 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260925 09:21:09.308457 12841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:09.310207 12827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:21:09.310417 12827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "eff90f98e7b94586b98067ca32d3788d"
format_stamp: "Formatted at 2026-09-25 09:21:09 on dist-test-slave-4wm5"
server_key: "8b71ee90ea378be8c9444d7c1a19c718"
server_key_iv: "78974b1ae3e4918138d5a4e53cb3aebc"
server_key_version: "encryptionkey@0"
I20260925 09:21:09.310660 12827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:09.336514 12827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:09.337500 12827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:09.337815 12827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:09.338913 12827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:09.340658 12827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:09.340790 12827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:09.340929 12827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:09.341017 12827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:09.374975 12827 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:41833
I20260925 09:21:09.375069 12953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:41833 every 8 connection(s)
I20260925 09:21:09.376267 12827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:21:09.386075 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12827
I20260925 09:21:09.386282 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:21:09.386921 3400 external_mini_cluster.cc:1428] Setting key a15bc4bac01da1c2e36e67563033ed32
I20260925 09:21:09.389734 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36029
--builtin_ntp_servers=127.3.82.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 09:21:09.390841 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:09.379497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48989 (local address 127.3.82.62:36029)
0925 09:21:09.380146 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:09.380160 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:09.381395 (+ 1235us) server_negotiation.cc:407] Connection header received
0925 09:21:09.383235 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:09.383240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:09.383328 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:09.383435 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:09.384636 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.385368 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.387598 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.388090 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.388254 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:09.389758 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:09.389766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:09.389772 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:09.389798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:09.389878 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:09.389899 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:09.389992 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:09.390292 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 09:21:09.390412 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":249,"threads_started":1}
I20260925 09:21:09.392239 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:09.380081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36029 (local address 127.3.82.1:48989)
0925 09:21:09.381232 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:09.381276 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:09.382966 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:09.383751 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:09.383761 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:09.383873 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:09.384465 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:09.384483 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.385591 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.385596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:09.387413 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:09.387428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.388356 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.388365 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:09.388615 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:09.389390 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:21:09.389393 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:09.389437 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:21:09.389529 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:09.390051 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:09.390061 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:09.390069 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 09:21:09.390282 (+ 213us) client_negotiation.cc:240] Negotiation successful
0925 09:21:09.390752 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":248,"threads_started":1}
I20260925 09:21:09.394038 12954 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36029
I20260925 09:21:09.394387 12954 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:09.395370 12954 heartbeater.cc:511] Master 127.3.82.62:36029 requested a full tablet report, sending...
I20260925 09:21:09.397728 12768 ts_manager.cc:194] Registered new tserver with Master: eff90f98e7b94586b98067ca32d3788d (127.3.82.1:41833)
I20260925 09:21:09.399457 12768 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.1:48989
W20260925 09:21:09.736146 12958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:09.736508 12958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:09.736675 12958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:09.744359 12958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:09.744539 12958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:09.744720 12958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:21:09.754429 12958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:09.755944 12958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:09.757995 12958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:09.773267 12963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:09.773290 12964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:09.773350 12966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:09.775108 12958 server_base.cc:1034] running on GCE node
I20260925 09:21:09.776058 12958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:09.777237 12958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:09.778527 12958 hybrid_clock.cc:648] HybridClock initialized: now 1790328069778482 us; error 42 us; skew 500 ppm
I20260925 09:21:09.779001 12958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:09.781903 12958 webserver.cc:460] Webserver started at http://127.3.82.2:34137/ using document root <none> and password file <none>
I20260925 09:21:09.782720 12958 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:09.782869 12958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:09.783214 12958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:09.786324 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "4e417255b54d4b3fa3a8f8cb8743a809"
format_stamp: "Formatted at 2026-09-25 09:21:09 on dist-test-slave-4wm5"
server_key: "909b56201e1b150de02bed0c78339f6d"
server_key_iv: "169bfd58ec62431630faf87191a20baf"
server_key_version: "encryptionkey@0"
I20260925 09:21:09.787225 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "4e417255b54d4b3fa3a8f8cb8743a809"
format_stamp: "Formatted at 2026-09-25 09:21:09 on dist-test-slave-4wm5"
server_key: "909b56201e1b150de02bed0c78339f6d"
server_key_iv: "169bfd58ec62431630faf87191a20baf"
server_key_version: "encryptionkey@0"
I20260925 09:21:09.793079 12958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 09:21:09.797312 12972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:09.799167 12958 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:21:09.799384 12958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "4e417255b54d4b3fa3a8f8cb8743a809"
format_stamp: "Formatted at 2026-09-25 09:21:09 on dist-test-slave-4wm5"
server_key: "909b56201e1b150de02bed0c78339f6d"
server_key_iv: "169bfd58ec62431630faf87191a20baf"
server_key_version: "encryptionkey@0"
I20260925 09:21:09.799630 12958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:09.828967 12958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:09.829967 12958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:09.830269 12958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:09.831312 12958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:09.833010 12958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:09.833133 12958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:09.833266 12958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:09.833357 12958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:09.869238 12958 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:44145
I20260925 09:21:09.869347 13084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:44145 every 8 connection(s)
I20260925 09:21:09.870700 12958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:21:09.877960 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 12958
I20260925 09:21:09.878160 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:21:09.878865 3400 external_mini_cluster.cc:1428] Setting key bab17c0a34313f27ca01c7265219b547
I20260925 09:21:09.882079 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36029
--builtin_ntp_servers=127.3.82.20:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 09:21:09.886111 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:09.873635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:37217 (local address 127.3.82.62:36029)
0925 09:21:09.873885 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:09.873892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:09.875501 (+ 1609us) server_negotiation.cc:407] Connection header received
0925 09:21:09.877753 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:09.877758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:09.877900 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:09.878038 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:09.879921 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.880783 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.882888 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.883372 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.883538 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:09.885142 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:09.885149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:09.885152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:09.885173 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:09.885208 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:09.885221 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:09.885330 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:09.885585 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 09:21:09.885715 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:21:09.887327 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:09.874285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36029 (local address 127.3.82.2:37217)
0925 09:21:09.875323 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:09.875370 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:09.877197 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:09.878366 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:09.878382 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:09.878542 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:09.879437 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:09.879470 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.880982 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.880988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:09.882709 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:09.882723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:09.883645 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:09.883654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:09.883916 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:09.884729 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:21:09.884732 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:09.884784 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:21:09.884887 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:09.885331 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:09.885346 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:09.885358 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 09:21:09.885579 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 09:21:09.886076 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":311,"threads_started":1}
I20260925 09:21:09.889014 13085 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36029
I20260925 09:21:09.889379 13085 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:09.890303 13085 heartbeater.cc:511] Master 127.3.82.62:36029 requested a full tablet report, sending...
I20260925 09:21:09.892354 12768 ts_manager.cc:194] Registered new tserver with Master: 4e417255b54d4b3fa3a8f8cb8743a809 (127.3.82.2:44145)
I20260925 09:21:09.893471 12768 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.2:37217
W20260925 09:21:10.240736 13088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:10.241098 13088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:10.241252 13088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:10.248960 13088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:10.249140 13088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:10.249329 13088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:21:10.260382 13088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:10.261775 13088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:10.263713 13088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:10.279037 13093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:10.278995 13094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:10.279211 13096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:10.280045 13088 server_base.cc:1034] running on GCE node
I20260925 09:21:10.280706 13088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:10.282094 13088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:10.283358 13088 hybrid_clock.cc:648] HybridClock initialized: now 1790328070283289 us; error 73 us; skew 500 ppm
I20260925 09:21:10.283746 13088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:10.286532 13088 webserver.cc:460] Webserver started at http://127.3.82.3:42585/ using document root <none> and password file <none>
I20260925 09:21:10.287369 13088 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:10.287881 13088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:10.288372 13088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:10.291882 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "a35005a0101a417db118432b0c929dc4"
format_stamp: "Formatted at 2026-09-25 09:21:10 on dist-test-slave-4wm5"
server_key: "1531441ac370031b1cfcb08f193b1691"
server_key_iv: "ec2ba9e388d25373176087b7b94b2ce7"
server_key_version: "encryptionkey@0"
I20260925 09:21:10.292876 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "a35005a0101a417db118432b0c929dc4"
format_stamp: "Formatted at 2026-09-25 09:21:10 on dist-test-slave-4wm5"
server_key: "1531441ac370031b1cfcb08f193b1691"
server_key_iv: "ec2ba9e388d25373176087b7b94b2ce7"
server_key_version: "encryptionkey@0"
I20260925 09:21:10.299304 13088 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:21:10.303949 13102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:10.305816 13088 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 09:21:10.306116 13088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "a35005a0101a417db118432b0c929dc4"
format_stamp: "Formatted at 2026-09-25 09:21:10 on dist-test-slave-4wm5"
server_key: "1531441ac370031b1cfcb08f193b1691"
server_key_iv: "ec2ba9e388d25373176087b7b94b2ce7"
server_key_version: "encryptionkey@0"
I20260925 09:21:10.306407 13088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:10.336655 13088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:10.337740 13088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:10.338057 13088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:10.339102 13088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:10.340685 13088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:10.340818 13088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:10.340955 13088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:10.341040 13088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:10.378475 13088 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:35771
I20260925 09:21:10.378523 13214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:35771 every 8 connection(s)
I20260925 09:21:10.379887 13088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:21:10.388890 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13088
I20260925 09:21:10.389030 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:21:10.389520 3400 external_mini_cluster.cc:1428] Setting key 3f1b6e30e95a293136d69aa533113cbb
I20260925 09:21:10.393625 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:10.382669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:46497 (local address 127.3.82.62:36029)
0925 09:21:10.382955 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:10.382963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:10.384587 (+ 1624us) server_negotiation.cc:407] Connection header received
0925 09:21:10.386450 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:10.386454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:10.386526 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:10.386611 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:10.387718 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.388555 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.390405 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.390977 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.391112 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:10.392747 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:10.392757 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:10.392762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:10.392785 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:10.392829 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:10.392849 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:10.392924 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:10.393225 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 09:21:10.393318 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260925 09:21:10.394640 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:10.383412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36029 (local address 127.3.82.3:46497)
0925 09:21:10.384403 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:10.384445 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:10.386146 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:10.386824 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:10.386834 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:10.386927 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:10.387541 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:10.387560 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.388722 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.388728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:10.390257 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:10.390269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.391212 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.391220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:10.391452 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:10.392269 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:21:10.392272 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:10.392326 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:21:10.392430 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:10.392959 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:10.392969 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:10.392980 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:21:10.393158 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 09:21:10.393478 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":352,"threads_started":1}
I20260925 09:21:10.396482 13215 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36029
I20260925 09:21:10.396891 13215 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:10.397717 13215 heartbeater.cc:511] Master 127.3.82.62:36029 requested a full tablet report, sending...
I20260925 09:21:10.399327 12768 ts_manager.cc:194] Registered new tserver with Master: a35005a0101a417db118432b0c929dc4 (127.3.82.3:35771)
I20260925 09:21:10.400286 12768 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.3:46497
I20260925 09:21:10.403579 12954 heartbeater.cc:503] Master 127.3.82.62:36029 was elected leader, sending a full tablet report...
I20260925 09:21:10.406085 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 09:21:10.412878 12956 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:10.411078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35112 (local address 127.3.82.62:36029)
0925 09:21:10.411263 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:10.411271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:10.411755 (+ 484us) server_negotiation.cc:407] Connection header received
0925 09:21:10.411958 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:10.411964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:10.412043 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:10.412168 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:10.412610 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35112: BlockingRecv error: recv got EOF from 127.0.0.1:35112 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:21:10.414317 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12827
I20260925 09:21:10.422953 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:10.415359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.3.82.1:41833)
0925 09:21:10.416041 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:10.416048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:10.416075 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:21:10.416132 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:10.416138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:10.416356 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:10.416499 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:10.417006 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.418107 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.419423 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.420001 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.420107 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:10.421259 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:10.421268 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:10.421282 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:10.422017 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:10.422073 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:10.422092 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:10.422183 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:10.422362 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 09:21:10.422476 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":297,"threads_started":1}
I20260925 09:21:10.424201 12949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:10.511003 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12827
I20260925 09:21:10.528025 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12958
I20260925 09:21:10.537127 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:10.529095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.3.82.2:44145)
0925 09:21:10.529687 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:10.529695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:10.529722 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:21:10.529785 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:10.529792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:10.530099 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:10.530281 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:10.530665 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.531734 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.533241 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.534076 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.534179 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:10.535300 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:10.535308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:10.535331 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:10.536142 (+ 811us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:10.536228 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:10.536248 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:10.536356 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:10.536539 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 09:21:10.536759 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":300,"threads_started":1}
I20260925 09:21:10.538780 13080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:10.622334 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12958
I20260925 09:21:10.639667 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13088
I20260925 09:21:10.649034 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:10.640929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57288 (local address 127.3.82.3:35771)
0925 09:21:10.641663 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:10.641671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:10.641703 (+ 32us) server_negotiation.cc:407] Connection header received
0925 09:21:10.641784 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:10.641793 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:10.642106 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:10.642259 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:10.642734 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.643847 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.645327 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:10.646070 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:10.646179 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:10.647438 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:10.647447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:10.647461 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:10.648090 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:10.648142 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:10.648174 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:10.648278 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:10.648473 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 09:21:10.648593 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":430,"threads_started":1}
I20260925 09:21:10.650532 13210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:10.735076 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13088
I20260925 09:21:10.751703 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 12737
I20260925 09:21:10.753101 12798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:10.859138 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 12737
2026-09-25T09:21:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2627 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T09:21:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:21:10Z Disabled control of system clock
I20260925 09:21:10.936008 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43171
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 09:21:11.284956 13234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:11.285341 13234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:11.285496 13234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:11.293118 13234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:21:11.293300 13234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:11.293392 13234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:11.293505 13234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:21:11.293582 13234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:21:11.303321 13234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43171
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:11.304735 13234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:11.306664 13234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:11.319321 13239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:11.319375 13240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:11.319383 13242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:11.320293 13234 server_base.cc:1034] running on GCE node
I20260925 09:21:11.321130 13234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:11.322741 13234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:11.323954 13234 hybrid_clock.cc:648] HybridClock initialized: now 1790328071323892 us; error 60 us; skew 500 ppm
I20260925 09:21:11.324282 13234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:11.334579 13234 webserver.cc:460] Webserver started at http://127.3.82.62:33637/ using document root <none> and password file <none>
I20260925 09:21:11.335408 13234 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:11.335511 13234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:11.335794 13234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:11.340760 13234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "5921f0c7701843f1b247b44a63cb81d1"
format_stamp: "Formatted at 2026-09-25 09:21:11 on dist-test-slave-4wm5"
server_key: "bb411399d64b964c62497982f5a33e9a"
server_key_iv: "8465892522c85e21cc4a0f87c2e494f7"
server_key_version: "encryptionkey@0"
I20260925 09:21:11.341653 13234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "5921f0c7701843f1b247b44a63cb81d1"
format_stamp: "Formatted at 2026-09-25 09:21:11 on dist-test-slave-4wm5"
server_key: "bb411399d64b964c62497982f5a33e9a"
server_key_iv: "8465892522c85e21cc4a0f87c2e494f7"
server_key_version: "encryptionkey@0"
I20260925 09:21:11.347510 13234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 09:21:11.351809 13248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:11.353408 13234 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260925 09:21:11.353660 13234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "5921f0c7701843f1b247b44a63cb81d1"
format_stamp: "Formatted at 2026-09-25 09:21:11 on dist-test-slave-4wm5"
server_key: "bb411399d64b964c62497982f5a33e9a"
server_key_iv: "8465892522c85e21cc4a0f87c2e494f7"
server_key_version: "encryptionkey@0"
I20260925 09:21:11.354048 13234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:11.361244 13234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:11.362282 13234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:11.362618 13234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:11.383580 13234 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:43171
I20260925 09:21:11.383637 13299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:43171 every 8 connection(s)
I20260925 09:21:11.385061 13234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:21:11.389716 13300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:21:11.391525 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13234
I20260925 09:21:11.391913 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:21:11.392583 3400 external_mini_cluster.cc:1428] Setting key 916b39b3fc61bc66486353a8df8914b0
I20260925 09:21:11.405181 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:11.395138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.3.82.62:43171)
0925 09:21:11.396679 (+ 1541us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:11.396704 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:11.396768 (+ 64us) server_negotiation.cc:407] Connection header received
0925 09:21:11.399651 (+ 2883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:11.399690 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:11.400382 (+ 692us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:11.400982 (+ 600us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:11.402688 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:11.402697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:11.402722 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:11.402743 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:11.402839 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:11.402869 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:11.403148 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:11.403384 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 09:21:11.403743 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":493,"threads_started":1}
I20260925 09:21:11.406750 13300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1: Bootstrap starting.
I20260925 09:21:11.410501 13300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1: Neither blocks nor log segments found. Creating new log.
I20260925 09:21:11.411865 13300 log.cc:824] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1: Log is configured to *not* fsync() on all Append() calls
I20260925 09:21:11.415138 13300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1: No bootstrap required, opened a new log
I20260925 09:21:11.423440 13300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5921f0c7701843f1b247b44a63cb81d1" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43171 } }
I20260925 09:21:11.423830 13300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:21:11.423905 13300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5921f0c7701843f1b247b44a63cb81d1, State: Initialized, Role: FOLLOWER
I20260925 09:21:11.424415 13300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5921f0c7701843f1b247b44a63cb81d1" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43171 } }
I20260925 09:21:11.424615 13300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:21:11.424712 13300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:21:11.424844 13300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:21:11.427152 13300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5921f0c7701843f1b247b44a63cb81d1" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43171 } }
I20260925 09:21:11.427651 13300 leader_election.cc:302] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5921f0c7701843f1b247b44a63cb81d1; no voters:
I20260925 09:21:11.428290 13300 leader_election.cc:288] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:21:11.428505 13305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:21:11.429761 13305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [term 1 LEADER]: Becoming Leader. State: Replica: 5921f0c7701843f1b247b44a63cb81d1, State: Running, Role: LEADER
I20260925 09:21:11.430493 13305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5921f0c7701843f1b247b44a63cb81d1" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43171 } }
I20260925 09:21:11.430956 13300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:21:11.433063 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5921f0c7701843f1b247b44a63cb81d1. Latest consensus state: current_term: 1 leader_uuid: "5921f0c7701843f1b247b44a63cb81d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5921f0c7701843f1b247b44a63cb81d1" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43171 } } }
I20260925 09:21:11.433398 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [sys.catalog]: This master's current role is: LEADER
I20260925 09:21:11.434042 13306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5921f0c7701843f1b247b44a63cb81d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5921f0c7701843f1b247b44a63cb81d1" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43171 } } }
I20260925 09:21:11.434330 13310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:21:11.434427 13306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1 [sys.catalog]: This master's current role is: LEADER
I20260925 09:21:11.441428 13310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:21:11.452826 13310 catalog_manager.cc:1347] Generated new cluster ID: 3571eeaff4ff429cb99236c1f505b4f2
I20260925 09:21:11.452966 13310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:21:11.465665 13310 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:21:11.466239 13310 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:21:11.478901 13310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5921f0c7701843f1b247b44a63cb81d1: Generated new TSK 0
I20260925 09:21:11.480072 13310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:21:11.487663 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43171
--builtin_ntp_servers=127.3.82.20:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 09:21:11.838943 13324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:11.839380 13324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:11.839561 13324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:11.847453 13324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:11.847628 13324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:11.847811 13324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:21:11.857686 13324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:11.859284 13324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:11.861289 13324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:11.875900 13329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:11.875984 13330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:11.875953 13332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:11.876995 13324 server_base.cc:1034] running on GCE node
I20260925 09:21:11.877734 13324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:11.878872 13324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:11.880132 13324 hybrid_clock.cc:648] HybridClock initialized: now 1790328071880086 us; error 39 us; skew 500 ppm
I20260925 09:21:11.880522 13324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:11.883226 13324 webserver.cc:460] Webserver started at http://127.3.82.1:39075/ using document root <none> and password file <none>
I20260925 09:21:11.884083 13324 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:11.884227 13324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:11.884577 13324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:11.887741 13324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "a7aba7b2b0de43c3bcc3ed19def5ccb9"
format_stamp: "Formatted at 2026-09-25 09:21:11 on dist-test-slave-4wm5"
server_key: "e0876770e97d80957dd4a94bda71f4f1"
server_key_iv: "ac855a08be5a5e1861e963c03693000b"
server_key_version: "encryptionkey@0"
I20260925 09:21:11.888664 13324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "a7aba7b2b0de43c3bcc3ed19def5ccb9"
format_stamp: "Formatted at 2026-09-25 09:21:11 on dist-test-slave-4wm5"
server_key: "e0876770e97d80957dd4a94bda71f4f1"
server_key_iv: "ac855a08be5a5e1861e963c03693000b"
server_key_version: "encryptionkey@0"
I20260925 09:21:11.894560 13324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 09:21:11.898875 13338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:11.900715 13324 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:21:11.900976 13324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "a7aba7b2b0de43c3bcc3ed19def5ccb9"
format_stamp: "Formatted at 2026-09-25 09:21:11 on dist-test-slave-4wm5"
server_key: "e0876770e97d80957dd4a94bda71f4f1"
server_key_iv: "ac855a08be5a5e1861e963c03693000b"
server_key_version: "encryptionkey@0"
I20260925 09:21:11.901279 13324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:11.908775 13324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:11.909912 13324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:11.910264 13324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:11.911423 13324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:11.913144 13324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:11.913295 13324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:11.913477 13324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:11.913583 13324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:11.954641 13324 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:36813
I20260925 09:21:11.954694 13450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:36813 every 8 connection(s)
I20260925 09:21:11.956307 13324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:21:11.965584 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13324
I20260925 09:21:11.965705 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:11.959245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:57247 (local address 127.3.82.62:43171)
0925 09:21:11.959712 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:11.959719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:11.961011 (+ 1292us) server_negotiation.cc:407] Connection header received
0925 09:21:11.962902 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:11.962907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:11.962968 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:11.963077 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:11.964686 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:11.964691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:11.964694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:11.964714 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:11.964762 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:11.964775 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:11.964897 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:11.965219 (+ 322us) server_negotiation.cc:299] Negotiation successful
0925 09:21:11.965342 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
I20260925 09:21:11.965862 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:21:11.966514 3400 external_mini_cluster.cc:1428] Setting key caad4d5ac357aabf57fe8361f05bdedb
I20260925 09:21:11.966807 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:11.959887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43171 (local address 127.3.82.1:57247)
0925 09:21:11.960832 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:11.960870 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:11.962557 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:11.963391 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:11.963401 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:11.963508 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:11.964275 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:21:11.964277 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:11.964322 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:21:11.964436 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:11.964952 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:11.964961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:11.964968 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:21:11.965219 (+ 251us) client_negotiation.cc:240] Negotiation successful
0925 09:21:11.965490 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":289,"threads_started":1}
I20260925 09:21:11.968538 13451 heartbeater.cc:347] Connected to a master server at 127.3.82.62:43171
I20260925 09:21:11.968890 13451 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:11.969524 13451 heartbeater.cc:511] Master 127.3.82.62:43171 requested a full tablet report, sending...
I20260925 09:21:11.969810 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43171
--builtin_ntp_servers=127.3.82.20:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 09:21:11.972070 13264 ts_manager.cc:194] Registered new tserver with Master: a7aba7b2b0de43c3bcc3ed19def5ccb9 (127.3.82.1:36813)
W20260925 09:21:12.335206 13455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:12.335510 13455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:12.335700 13455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:12.343760 13455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:12.343915 13455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:12.344094 13455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:21:12.353818 13455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:12.355248 13455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:12.357069 13455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:12.371138 13460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:12.371167 13461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:12.371136 13463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:12.372130 13455 server_base.cc:1034] running on GCE node
I20260925 09:21:12.372924 13455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:12.374114 13455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:12.375388 13455 hybrid_clock.cc:648] HybridClock initialized: now 1790328072375351 us; error 40 us; skew 500 ppm
I20260925 09:21:12.375818 13455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:12.378855 13455 webserver.cc:460] Webserver started at http://127.3.82.2:35331/ using document root <none> and password file <none>
I20260925 09:21:12.379698 13455 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:12.379845 13455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:12.380213 13455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:12.383700 13455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "629897ac5c194b98a3f1e03e94c0a477"
format_stamp: "Formatted at 2026-09-25 09:21:12 on dist-test-slave-4wm5"
server_key: "0b1fef6cff605c75dfd001d1be6f0c84"
server_key_iv: "2b3763f06015c070e707cf2248abdeb5"
server_key_version: "encryptionkey@0"
I20260925 09:21:12.384791 13455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "629897ac5c194b98a3f1e03e94c0a477"
format_stamp: "Formatted at 2026-09-25 09:21:12 on dist-test-slave-4wm5"
server_key: "0b1fef6cff605c75dfd001d1be6f0c84"
server_key_iv: "2b3763f06015c070e707cf2248abdeb5"
server_key_version: "encryptionkey@0"
I20260925 09:21:12.392143 13455 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260925 09:21:12.397365 13469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:12.399379 13455 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:21:12.399622 13455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "629897ac5c194b98a3f1e03e94c0a477"
format_stamp: "Formatted at 2026-09-25 09:21:12 on dist-test-slave-4wm5"
server_key: "0b1fef6cff605c75dfd001d1be6f0c84"
server_key_iv: "2b3763f06015c070e707cf2248abdeb5"
server_key_version: "encryptionkey@0"
I20260925 09:21:12.399925 13455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:12.406806 13455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:12.407830 13455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:12.408178 13455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:12.409286 13455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:12.411082 13455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:12.411222 13455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:12.411379 13455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:12.411478 13455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:12.451303 13455 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:37821
I20260925 09:21:12.451417 13581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:37821 every 8 connection(s)
I20260925 09:21:12.452907 13455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:21:12.457798 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13455
I20260925 09:21:12.458073 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:21:12.458680 3400 external_mini_cluster.cc:1428] Setting key 2135c546d54a765ff5fa2bfb944526ae
I20260925 09:21:12.461999 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43171
--builtin_ntp_servers=127.3.82.20:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 09:21:12.463541 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:12.456273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:35827 (local address 127.3.82.62:43171)
0925 09:21:12.456486 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:12.456492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:12.458406 (+ 1914us) server_negotiation.cc:407] Connection header received
0925 09:21:12.460620 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:12.460626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:12.460715 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:12.460833 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:12.462600 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:12.462607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:12.462611 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:12.462634 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:12.462691 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:12.462709 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:12.462871 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:12.463124 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 09:21:12.463240 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 09:21:12.464730 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:12.456852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43171 (local address 127.3.82.2:35827)
0925 09:21:12.458112 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:12.458246 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:12.460321 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:12.461103 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:12.461113 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:12.461231 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:12.462078 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:21:12.462081 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:12.462175 (+ 94us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:21:12.462334 (+ 159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:12.462832 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:12.462844 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:12.462854 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:21:12.463226 (+ 372us) client_negotiation.cc:240] Negotiation successful
0925 09:21:12.463554 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":405,"threads_started":1}
I20260925 09:21:12.466780 13582 heartbeater.cc:347] Connected to a master server at 127.3.82.62:43171
I20260925 09:21:12.467105 13582 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:12.467895 13582 heartbeater.cc:511] Master 127.3.82.62:43171 requested a full tablet report, sending...
I20260925 09:21:12.469677 13264 ts_manager.cc:194] Registered new tserver with Master: 629897ac5c194b98a3f1e03e94c0a477 (127.3.82.2:37821)
W20260925 09:21:12.816416 13585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:12.816761 13585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:12.816922 13585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:12.824493 13585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:12.824621 13585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:12.824822 13585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:21:12.834784 13585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:12.836186 13585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:12.838274 13585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:12.852780 13591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:12.852726 13593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:12.852833 13590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:12.853857 13585 server_base.cc:1034] running on GCE node
I20260925 09:21:12.854571 13585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:12.855758 13585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:12.857012 13585 hybrid_clock.cc:648] HybridClock initialized: now 1790328072856956 us; error 45 us; skew 500 ppm
I20260925 09:21:12.857445 13585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:21:12.860384 13585 webserver.cc:460] Webserver started at http://127.3.82.3:38065/ using document root <none> and password file <none>
I20260925 09:21:12.861199 13585 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:12.861310 13585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:12.861657 13585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:12.864880 13585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "1c7609445cd64c4898d198b85324a9f4"
format_stamp: "Formatted at 2026-09-25 09:21:12 on dist-test-slave-4wm5"
server_key: "b0194b2d21866e16591c0c522c63da15"
server_key_iv: "40789d5b89f03d42f4b12551d1c2f04f"
server_key_version: "encryptionkey@0"
I20260925 09:21:12.865749 13585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "1c7609445cd64c4898d198b85324a9f4"
format_stamp: "Formatted at 2026-09-25 09:21:12 on dist-test-slave-4wm5"
server_key: "b0194b2d21866e16591c0c522c63da15"
server_key_iv: "40789d5b89f03d42f4b12551d1c2f04f"
server_key_version: "encryptionkey@0"
I20260925 09:21:12.871492 13585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:21:12.875283 13599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:12.876775 13585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:21:12.876971 13585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "1c7609445cd64c4898d198b85324a9f4"
format_stamp: "Formatted at 2026-09-25 09:21:12 on dist-test-slave-4wm5"
server_key: "b0194b2d21866e16591c0c522c63da15"
server_key_iv: "40789d5b89f03d42f4b12551d1c2f04f"
server_key_version: "encryptionkey@0"
I20260925 09:21:12.877244 13585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:12.883461 13585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:12.884526 13585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:12.884822 13585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:12.886013 13585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:12.887531 13585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:12.887625 13585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:12.887769 13585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:12.887892 13585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:12.923606 13585 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:43987
I20260925 09:21:12.923661 13711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:43987 every 8 connection(s)
I20260925 09:21:12.925014 13585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:21:12.928915 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13585
I20260925 09:21:12.929152 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:21:12.929674 3400 external_mini_cluster.cc:1428] Setting key 9a3361070bac443c733626780649f03f
I20260925 09:21:12.934664 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:12.927941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:60585 (local address 127.3.82.62:43171)
0925 09:21:12.928168 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:12.928176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:12.929975 (+ 1799us) server_negotiation.cc:407] Connection header received
0925 09:21:12.932304 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:12.932309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:12.932376 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:12.932501 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:12.934044 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:12.934048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:12.934051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:12.934068 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:12.934099 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:12.934111 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:12.934205 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:12.934392 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 09:21:12.934445 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:21:12.935716 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:12.928577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43171 (local address 127.3.82.3:60585)
0925 09:21:12.929726 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:12.929803 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:12.932003 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:12.932856 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:12.932871 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:12.932975 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:12.933689 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:21:12.933691 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:12.933730 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:21:12.933879 (+ 149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:12.934204 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:12.934211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:12.934216 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 09:21:12.934435 (+ 219us) client_negotiation.cc:240] Negotiation successful
0925 09:21:12.934644 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":386,"threads_started":1}
I20260925 09:21:12.937093 13712 heartbeater.cc:347] Connected to a master server at 127.3.82.62:43171
I20260925 09:21:12.937459 13712 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:12.938081 13712 heartbeater.cc:511] Master 127.3.82.62:43171 requested a full tablet report, sending...
I20260925 09:21:12.939637 13264 ts_manager.cc:194] Registered new tserver with Master: 1c7609445cd64c4898d198b85324a9f4 (127.3.82.3:43987)
I20260925 09:21:12.946065 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 09:21:12.952996 13453 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:21:12.951204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.3.82.62:43171)
0925 09:21:12.951430 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:12.951435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:12.952036 (+ 601us) server_negotiation.cc:407] Connection header received
0925 09:21:12.952191 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:12.952195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:12.952252 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:12.952355 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:12.952717 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51758: BlockingRecv error: recv got EOF from 127.0.0.1:51758 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:21:12.953613 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13324
I20260925 09:21:12.961805 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:12.954706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.3.82.1:36813)
0925 09:21:12.955381 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:12.955388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:12.959096 (+ 3708us) server_negotiation.cc:407] Connection header received
0925 09:21:12.959365 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:12.959373 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:12.959663 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:12.959833 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:12.961013 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:12.961020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:12.961030 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:12.961043 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:12.961082 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:12.961095 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:12.961301 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:12.961369 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 09:21:12.961480 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":406,"threads_started":1}
I20260925 09:21:12.963073 13446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:13.047515 13451 heartbeater.cc:503] Master 127.3.82.62:43171 was elected leader, sending a full tablet report...
I20260925 09:21:13.048794 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13324
W20260925 09:21:13.065320 13252 connection.cc:535] server connection from 127.3.82.1:57247 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:21:13.065609 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13455
I20260925 09:21:13.069621 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:13.066802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.3.82.2:37821)
0925 09:21:13.067360 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:13.067368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:13.067396 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:21:13.067462 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:13.067468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:13.067708 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:13.067839 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:13.068855 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:13.068862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:13.068874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:13.068892 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:13.068935 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:13.068950 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:13.069059 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:13.069196 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 09:21:13.069327 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":249,"threads_started":1}
I20260925 09:21:13.071133 13577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:13.160514 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13455
I20260925 09:21:13.176617 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13585
I20260925 09:21:13.180969 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:13.177687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.3.82.3:43987)
0925 09:21:13.178404 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:13.178412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:13.178442 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:21:13.178509 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:13.178515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:13.178740 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:13.178923 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:13.180137 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:13.180144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:13.180155 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:21:13.180172 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:13.180210 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:21:13.180225 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:13.180449 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:13.180548 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 09:21:13.180660 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":381,"threads_started":1}
I20260925 09:21:13.182396 13707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:13.266897 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13585
I20260925 09:21:13.282925 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13234
I20260925 09:21:13.284430 13295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:13.390533 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13234
2026-09-25T09:21:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2536 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:21:13 dist-test-slave-4wm5 krb5kdc[13730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:21:13 dist-test-slave-4wm5 krb5kdc[13730](info): set up 2 sockets
Sep 25 09:21:13 dist-test-slave-4wm5 krb5kdc[13730](info): commencing operation
krb5kdc: starting...
W20260925 09:21:14.530844 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:21:14 dist-test-slave-4wm5 krb5kdc[13730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:21:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:21:14.790107 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43155
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:21:15.140319 13743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:15.140650 13743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:15.140838 13743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:15.150190 13743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:21:15.150360 13743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:15.150451 13743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:15.150542 13743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:21:15.150609 13743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:21:15.163573 13743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:15.165148 13743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:15.166885 13743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:15.181022 13748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:15.180985 13749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:15.181056 13751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:15.182487 13743 server_base.cc:1034] running on GCE node
I20260925 09:21:15.183409 13743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:15.184993 13743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:15.186268 13743 hybrid_clock.cc:648] HybridClock initialized: now 1790328075186200 us; error 87 us; skew 500 ppm
I20260925 09:21:15.186728 13743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:21:15 dist-test-slave-4wm5 krb5kdc[13730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328075, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:21:15.190713 13743 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:21:15.192529 13743 webserver.cc:460] Webserver started at http://127.3.82.62:34195/ using document root <none> and password file <none>
I20260925 09:21:15.193320 13743 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:15.193481 13743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:15.193809 13743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:15.197045 13743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "2177e86c46b643f8980cd55abf33fa35"
format_stamp: "Formatted at 2026-09-25 09:21:15 on dist-test-slave-4wm5"
server_key: "818c22043056826d30d2894324cc4a40"
server_key_iv: "ee6e53f3ceaa3697274c01e653021818"
server_key_version: "encryptionkey@0"
I20260925 09:21:15.197952 13743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "2177e86c46b643f8980cd55abf33fa35"
format_stamp: "Formatted at 2026-09-25 09:21:15 on dist-test-slave-4wm5"
server_key: "818c22043056826d30d2894324cc4a40"
server_key_iv: "ee6e53f3ceaa3697274c01e653021818"
server_key_version: "encryptionkey@0"
I20260925 09:21:15.204473 13743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260925 09:21:15.208432 13758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:15.210280 13743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:21:15.210500 13743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2177e86c46b643f8980cd55abf33fa35"
format_stamp: "Formatted at 2026-09-25 09:21:15 on dist-test-slave-4wm5"
server_key: "818c22043056826d30d2894324cc4a40"
server_key_iv: "ee6e53f3ceaa3697274c01e653021818"
server_key_version: "encryptionkey@0"
I20260925 09:21:15.210743 13743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:15.225773 13743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:15.231231 13743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:15.231624 13743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:15.253280 13809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:43155 every 8 connection(s)
I20260925 09:21:15.253274 13743 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:43155
I20260925 09:21:15.255041 13743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:21:15.255533 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13743
I20260925 09:21:15.255788 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:21:15.256721 3400 external_mini_cluster.cc:1428] Setting key aba6082e1a7ca8471af8a3690ee6606a
I20260925 09:21:15.260120 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:21:15 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:21:15.274833 13810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35: Bootstrap starting.
I20260925 09:21:15.278571 13810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35: Neither blocks nor log segments found. Creating new log.
I20260925 09:21:15.280021 13810 log.cc:824] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35: Log is configured to *not* fsync() on all Append() calls
I20260925 09:21:15.283041 13813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:15.258965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.3.82.62:43155)
0925 09:21:15.259868 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:15.259882 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:15.259925 (+ 43us) server_negotiation.cc:407] Connection header received
0925 09:21:15.262103 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:15.262127 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:15.262540 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:15.262893 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:15.264798 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:15.266320 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:15.268294 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:15.269041 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:15.269199 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:15.274252 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:15.274261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:15.274279 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:15.274328 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:15.277594 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:15.278580 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:15.278590 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:15.278605 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:15.278952 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:15.279445 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:15.279450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:15.279461 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:15.280050 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:15.280386 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:15.280930 (+ 544us) server_negotiation.cc:299] Negotiation successful
0925 09:21:15.281437 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":468,"threads_started":1}
I20260925 09:21:15.283839 13810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35: No bootstrap required, opened a new log
I20260925 09:21:15.291718 13810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2177e86c46b643f8980cd55abf33fa35" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43155 } }
I20260925 09:21:15.292151 13810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:21:15.292268 13810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2177e86c46b643f8980cd55abf33fa35, State: Initialized, Role: FOLLOWER
I20260925 09:21:15.293044 13810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2177e86c46b643f8980cd55abf33fa35" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43155 } }
I20260925 09:21:15.293346 13810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:21:15.293516 13810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:21:15.293713 13810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:21:15.296015 13810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2177e86c46b643f8980cd55abf33fa35" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43155 } }
I20260925 09:21:15.296517 13810 leader_election.cc:302] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2177e86c46b643f8980cd55abf33fa35; no voters:
I20260925 09:21:15.297099 13810 leader_election.cc:288] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:21:15.297274 13815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:21:15.298007 13815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [term 1 LEADER]: Becoming Leader. State: Replica: 2177e86c46b643f8980cd55abf33fa35, State: Running, Role: LEADER
I20260925 09:21:15.298671 13815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2177e86c46b643f8980cd55abf33fa35" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43155 } }
I20260925 09:21:15.299396 13810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:21:15.301496 13816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2177e86c46b643f8980cd55abf33fa35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2177e86c46b643f8980cd55abf33fa35" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43155 } } }
I20260925 09:21:15.302138 13816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [sys.catalog]: This master's current role is: LEADER
I20260925 09:21:15.302079 13817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2177e86c46b643f8980cd55abf33fa35. Latest consensus state: current_term: 1 leader_uuid: "2177e86c46b643f8980cd55abf33fa35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2177e86c46b643f8980cd55abf33fa35" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43155 } } }
I20260925 09:21:15.302409 13817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35 [sys.catalog]: This master's current role is: LEADER
I20260925 09:21:15.304416 13820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:21:15.309736 13820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:21:15.333580 13820 catalog_manager.cc:1347] Generated new cluster ID: 8ef619558fcc49c0a9f7ff45c5e5fbf7
I20260925 09:21:15.333848 13820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:21:15.376327 13820 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:21:15.378254 13820 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:21:15.390240 13820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35: Generated new TSK 0
I20260925 09:21:15.391409 13820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:21:15.472162 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43155
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:21:15.836227 13836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:15.836544 13836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:15.836705 13836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:15.844691 13836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:15.844859 13836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:15.845077 13836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:21:15.856627 13836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:15.858167 13836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:15.859715 13836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:15.874974 13841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:15.875034 13842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:15.875053 13844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:15.880198 13836 server_base.cc:1034] running on GCE node
I20260925 09:21:15.880952 13836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:15.882118 13836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:15.883582 13836 hybrid_clock.cc:648] HybridClock initialized: now 1790328075883520 us; error 77 us; skew 500 ppm
I20260925 09:21:15.884099 13836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:21:15 dist-test-slave-4wm5 krb5kdc[13730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328075, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:21:15.888456 13836 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:21:15.890738 13836 webserver.cc:460] Webserver started at http://127.3.82.1:38267/ using document root <none> and password file <none>
I20260925 09:21:15.891558 13836 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:15.891732 13836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:15.892076 13836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:15.895645 13836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "f62b9b8c17b64cb495adb831d9156f69"
format_stamp: "Formatted at 2026-09-25 09:21:15 on dist-test-slave-4wm5"
server_key: "7f578912cbf06725ddf4d7a6314b43ab"
server_key_iv: "06a13c5083a6de3ed1eec8cd6e5ec4d2"
server_key_version: "encryptionkey@0"
I20260925 09:21:15.896651 13836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "f62b9b8c17b64cb495adb831d9156f69"
format_stamp: "Formatted at 2026-09-25 09:21:15 on dist-test-slave-4wm5"
server_key: "7f578912cbf06725ddf4d7a6314b43ab"
server_key_iv: "06a13c5083a6de3ed1eec8cd6e5ec4d2"
server_key_version: "encryptionkey@0"
I20260925 09:21:15.902983 13836 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260925 09:21:15.907296 13851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:15.909509 13836 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:21:15.909770 13836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "f62b9b8c17b64cb495adb831d9156f69"
format_stamp: "Formatted at 2026-09-25 09:21:15 on dist-test-slave-4wm5"
server_key: "7f578912cbf06725ddf4d7a6314b43ab"
server_key_iv: "06a13c5083a6de3ed1eec8cd6e5ec4d2"
server_key_version: "encryptionkey@0"
I20260925 09:21:15.910149 13836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:15.926096 13836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:15.931685 13836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:15.932039 13836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:15.933166 13836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:15.934753 13836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:15.934881 13836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:15.935045 13836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:15.935119 13836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:15.972518 13836 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:46563
I20260925 09:21:15.972579 13963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:46563 every 8 connection(s)
I20260925 09:21:15.973985 13836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:21:15.981726 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13836
I20260925 09:21:15.982110 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:21:15.982755 3400 external_mini_cluster.cc:1428] Setting key 557da338e1da4d0ff7defd8c1b616981
Sep 25 09:21:15 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328075, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:21:15.999266 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:15.977292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:38191 (local address 127.3.82.62:43155)
0925 09:21:15.977917 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:15.977932 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:15.979794 (+ 1862us) server_negotiation.cc:407] Connection header received
0925 09:21:15.981728 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:15.981734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:15.982193 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:15.982357 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:15.984291 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:15.985299 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:15.987327 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:15.987759 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:15.987915 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:15.992331 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:15.992337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:15.992341 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:15.992379 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:15.995652 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:15.996652 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:15.996657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:15.996659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:15.996796 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:15.997292 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:15.997304 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:15.997307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:15.997600 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:15.997778 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:15.998611 (+ 833us) server_negotiation.cc:299] Negotiation successful
0925 09:21:15.998867 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":251,"threads_started":1}
I20260925 09:21:16.000433 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:15.978361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43155 (local address 127.3.82.1:38191)
0925 09:21:15.979501 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:15.979550 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:15.981373 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:15.982652 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:15.982662 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:15.983359 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:15.984105 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:15.984126 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:15.985498 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:15.985504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:15.987129 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:15.987149 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:15.988042 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:15.988051 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:15.988312 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:15.989136 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:15.989160 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:15.992021 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:15.995882 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:21:15.995894 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:21:15.995920 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:21:15.996479 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:21:15.996928 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:21:15.996933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:21:15.996937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:21:15.997153 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:21:15.997756 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:15.997765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:15.998199 (+ 434us) client_negotiation.cc:769] Sending connection context
0925 09:21:15.998494 (+ 295us) client_negotiation.cc:240] Negotiation successful
0925 09:21:15.999083 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":370,"threads_started":1}
I20260925 09:21:16.002328 13964 heartbeater.cc:347] Connected to a master server at 127.3.82.62:43155
I20260925 09:21:16.002774 13964 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:16.003846 13964 heartbeater.cc:511] Master 127.3.82.62:43155 requested a full tablet report, sending...
I20260925 09:21:16.006856 13775 ts_manager.cc:194] Registered new tserver with Master: f62b9b8c17b64cb495adb831d9156f69 (127.3.82.1:46563)
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:21:16.009413 13775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:38191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:21:16.046618 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43155
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:21:16.407989 13970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:16.408293 13970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:16.408464 13970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:16.416769 13970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:16.416903 13970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:16.417097 13970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:21:16.428865 13970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:16.430706 13970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:16.432497 13970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:16.449654 13975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:16.449597 13978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:16.449625 13976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:16.450752 13970 server_base.cc:1034] running on GCE node
I20260925 09:21:16.451689 13970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:16.452888 13970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:16.454118 13970 hybrid_clock.cc:648] HybridClock initialized: now 1790328076454060 us; error 48 us; skew 500 ppm
I20260925 09:21:16.454540 13970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:21:16 dist-test-slave-4wm5 krb5kdc[13730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328076, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:21:16.464735 13970 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:21:16.467121 13970 webserver.cc:460] Webserver started at http://127.3.82.2:37333/ using document root <none> and password file <none>
I20260925 09:21:16.468015 13970 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:16.468209 13970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:16.468580 13970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:16.472321 13970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "25c7a6bb15bf4b908972c90d535085e2"
format_stamp: "Formatted at 2026-09-25 09:21:16 on dist-test-slave-4wm5"
server_key: "e044f8172fd14bc62127836eb94aa308"
server_key_iv: "d787b11e332b74dbed9ce94ccc74c136"
server_key_version: "encryptionkey@0"
I20260925 09:21:16.473364 13970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "25c7a6bb15bf4b908972c90d535085e2"
format_stamp: "Formatted at 2026-09-25 09:21:16 on dist-test-slave-4wm5"
server_key: "e044f8172fd14bc62127836eb94aa308"
server_key_iv: "d787b11e332b74dbed9ce94ccc74c136"
server_key_version: "encryptionkey@0"
I20260925 09:21:16.480547 13970 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260925 09:21:16.485473 13985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:16.487769 13970 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:21:16.488121 13970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "25c7a6bb15bf4b908972c90d535085e2"
format_stamp: "Formatted at 2026-09-25 09:21:16 on dist-test-slave-4wm5"
server_key: "e044f8172fd14bc62127836eb94aa308"
server_key_iv: "d787b11e332b74dbed9ce94ccc74c136"
server_key_version: "encryptionkey@0"
I20260925 09:21:16.488389 13970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:16.513195 13970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:16.518993 13970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:16.519330 13970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:16.520485 13970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:16.522037 13970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:16.522156 13970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:16.522325 13970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:16.522403 13970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:16.561491 13970 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:45337
I20260925 09:21:16.561511 14097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:45337 every 8 connection(s)
I20260925 09:21:16.563030 13970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:21:16.563555 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 13970
I20260925 09:21:16.563781 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:21:16.564376 3400 external_mini_cluster.cc:1428] Setting key ca6ed23d05fb61ec0b0da94493608922
Sep 25 09:21:16 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328076, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:21:16.590097 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:16.566078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:60481 (local address 127.3.82.62:43155)
0925 09:21:16.566619 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:16.566624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:16.568134 (+ 1510us) server_negotiation.cc:407] Connection header received
0925 09:21:16.571342 (+ 3208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:16.571357 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:16.571478 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:16.571598 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:16.574028 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:16.574852 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:16.577059 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:16.578099 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:16.578265 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:16.582817 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:16.582824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:16.582828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:16.582874 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:16.585764 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:16.587075 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:16.587080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:16.587083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:16.587223 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:16.587913 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:16.587917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:16.587919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:16.588212 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:16.588360 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:16.589533 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0925 09:21:16.589738 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":277,"threads_started":1}
I20260925 09:21:16.591928 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:16.566955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43155 (local address 127.3.82.2:60481)
0925 09:21:16.567938 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:16.567998 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:16.570946 (+ 2948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:16.571955 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:16.571970 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:16.572844 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:16.573742 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:16.573771 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:16.575054 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:16.575063 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:16.576810 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:16.576824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:16.578383 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:16.578398 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:16.578618 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:16.579480 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:16.579509 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:16.582389 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:16.586007 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:21:16.586023 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:21:16.586055 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:21:16.586908 (+ 853us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:21:16.587429 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:21:16.587439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:21:16.587446 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:21:16.587774 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:21:16.588403 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:16.588417 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:16.589032 (+ 615us) client_negotiation.cc:769] Sending connection context
0925 09:21:16.589445 (+ 413us) client_negotiation.cc:240] Negotiation successful
0925 09:21:16.590181 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":383,"threads_started":1}
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:21:16.593943 14098 heartbeater.cc:347] Connected to a master server at 127.3.82.62:43155
I20260925 09:21:16.594349 14098 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:16.595309 14098 heartbeater.cc:511] Master 127.3.82.62:43155 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:21:16.597119 13775 ts_manager.cc:194] Registered new tserver with Master: 25c7a6bb15bf4b908972c90d535085e2 (127.3.82.2:45337)
I20260925 09:21:16.598141 13775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:60481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:21:16.628153 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43155
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:21:16.974973 14104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:21:16.975313 14104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:21:16.975487 14104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:21:16.983227 14104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:21:16.983392 14104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:21:16.983594 14104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:21:16.993935 14104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:43155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:21:16.995388 14104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:21:16.996769 14104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:21:17.012516 14110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:17.012470 14112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:21:17.012566 14109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:21:17.012846 14104 server_base.cc:1034] running on GCE node
I20260925 09:21:17.013484 13964 heartbeater.cc:503] Master 127.3.82.62:43155 was elected leader, sending a full tablet report...
I20260925 09:21:17.013944 14104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:21:17.015210 14104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:21:17.016522 14104 hybrid_clock.cc:648] HybridClock initialized: now 1790328077016454 us; error 56 us; skew 500 ppm
I20260925 09:21:17.016862 14104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:21:17 dist-test-slave-4wm5 krb5kdc[13730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328077, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:21:17.020406 14104 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:21:17.022233 14104 webserver.cc:460] Webserver started at http://127.3.82.3:37513/ using document root <none> and password file <none>
I20260925 09:21:17.022957 14104 fs_manager.cc:359] Metadata directory not provided
I20260925 09:21:17.023108 14104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:21:17.023437 14104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:21:17.026528 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "ae124341d35442bf8cab9a57b5ba529c"
format_stamp: "Formatted at 2026-09-25 09:21:17 on dist-test-slave-4wm5"
server_key: "ad5cb24cd9c6490c30cbedcbcf869e77"
server_key_iv: "8d0726c06833e18080461cef44c5e7b5"
server_key_version: "encryptionkey@0"
I20260925 09:21:17.027364 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "ae124341d35442bf8cab9a57b5ba529c"
format_stamp: "Formatted at 2026-09-25 09:21:17 on dist-test-slave-4wm5"
server_key: "ad5cb24cd9c6490c30cbedcbcf869e77"
server_key_iv: "8d0726c06833e18080461cef44c5e7b5"
server_key_version: "encryptionkey@0"
I20260925 09:21:17.033272 14104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:21:17.037493 14119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:17.039151 14104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 09:21:17.039407 14104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "ae124341d35442bf8cab9a57b5ba529c"
format_stamp: "Formatted at 2026-09-25 09:21:17 on dist-test-slave-4wm5"
server_key: "ad5cb24cd9c6490c30cbedcbcf869e77"
server_key_iv: "8d0726c06833e18080461cef44c5e7b5"
server_key_version: "encryptionkey@0"
I20260925 09:21:17.039635 14104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:21:17.055487 14104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:21:17.060171 14104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:21:17.060501 14104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:21:17.061579 14104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:21:17.063190 14104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:21:17.063303 14104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:17.063482 14104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:21:17.063562 14104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:21:17.100214 14104 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:33655
I20260925 09:21:17.100306 14231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:33655 every 8 connection(s)
I20260925 09:21:17.101604 14104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:21:17.104969 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 14104
I20260925 09:21:17.105172 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:21:17.105583 3400 external_mini_cluster.cc:1428] Setting key 87769866f3ec63261ae1c7e1e5acb45d
Sep 25 09:21:17 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328077, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:21:17.125231 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.104397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:55301 (local address 127.3.82.62:43155)
0925 09:21:17.104951 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.104957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.106039 (+ 1082us) server_negotiation.cc:407] Connection header received
0925 09:21:17.108152 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.108156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.108250 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.108349 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:17.110056 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.111031 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.112792 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.113218 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.113331 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.117456 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:17.117462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:17.117465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:17.117506 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:17.121026 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.122355 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.122361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.122364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.122581 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.123157 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.123163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.123166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.123592 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:17.123805 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.124596 (+ 791us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.124854 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":287,"threads_started":1}
I20260925 09:21:17.126644 14235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.104936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43155 (local address 127.3.82.3:55301)
0925 09:21:17.105851 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:17.105902 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:17.107808 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:17.108618 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:17.108632 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:17.109222 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:21:17.109887 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.109918 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.111182 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.111186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.112642 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.112656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.113455 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.113461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.113720 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.114529 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:21:17.114551 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:21:17.117182 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:21:17.121267 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:21:17.121281 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:21:17.121302 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:21:17.122171 (+ 869us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:21:17.122747 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:21:17.122753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:21:17.122757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:21:17.123026 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:21:17.123752 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:21:17.123760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:21:17.124192 (+ 432us) client_negotiation.cc:769] Sending connection context
0925 09:21:17.124485 (+ 293us) client_negotiation.cc:240] Negotiation successful
0925 09:21:17.125053 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":362,"threads_started":1}
I20260925 09:21:17.128563 14232 heartbeater.cc:347] Connected to a master server at 127.3.82.62:43155
I20260925 09:21:17.128989 14232 heartbeater.cc:464] Registering TS with master...
I20260925 09:21:17.129779 14232 heartbeater.cc:511] Master 127.3.82.62:43155 requested a full tablet report, sending...
I20260925 09:21:17.131389 13775 ts_manager.cc:194] Registered new tserver with Master: ae124341d35442bf8cab9a57b5ba529c (127.3.82.3:33655)
I20260925 09:21:17.132339 13775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:55301
I20260925 09:21:17.135402 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:21:17.156243 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.139967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.3.82.62:43155)
0925 09:21:17.140236 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.140246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.140615 (+ 369us) server_negotiation.cc:407] Connection header received
0925 09:21:17.140915 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.140919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.140983 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.141093 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:17.143217 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.144168 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.146509 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.147269 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.147432 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.150349 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:17.150363 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:17.150366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:17.150407 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:17.153299 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.154099 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.154104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.154108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.154243 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.154654 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.154659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.154662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.155048 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:17.155304 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.155670 (+ 366us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.155912 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:21:17.161506 13775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:21:17.169368 13775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:21:17.196291 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.184392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.3.82.3:33655)
0925 09:21:17.185631 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.185642 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.185876 (+ 234us) server_negotiation.cc:407] Connection header received
0925 09:21:17.186766 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.186782 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.187129 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.187405 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:21:17.188722 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.190305 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.193327 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.194932 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.195078 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.195332 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.195709 (+ 377us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.195897 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":912,"threads_started":1}
I20260925 09:21:17.196573 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.185375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.3.82.2:45337)
0925 09:21:17.186124 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.186135 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.187020 (+ 885us) server_negotiation.cc:407] Connection header received
0925 09:21:17.187120 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.187129 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.187374 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.187563 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:21:17.188987 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.190703 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.193396 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.195015 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.195164 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.195389 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.196003 (+ 614us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.196210 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":427,"threads_started":1}
I20260925 09:21:17.196458 14245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.184157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:33655 (local address 127.0.0.1:41548)
0925 09:21:17.185037 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:17.185092 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:17.186323 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:17.187840 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:17.187852 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:17.187947 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:21:17.188467 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.188496 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.190485 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.190491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.193122 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.193137 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.195151 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.195158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.195385 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.195457 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 09:21:17.195638 (+ 181us) client_negotiation.cc:240] Negotiation successful
0925 09:21:17.195889 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":72,"spinlock_wait_cycles":62976,"thread_start_us":372,"threads_started":1}
I20260925 09:21:17.197400 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.184935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:45337 (local address 127.0.0.1:46710)
0925 09:21:17.186750 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:17.186767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:17.186896 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:17.187850 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:17.187858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:17.187928 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:21:17.188813 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.188823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.190900 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.190907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.193107 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.193122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.195556 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.195565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.195781 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.195817 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:21:17.196855 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0925 09:21:17.197032 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1696,"mutex_wait_us":64,"spinlock_wait_cycles":24320,"thread_start_us":1220,"threads_started":1}
I20260925 09:21:17.197631 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.186141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.3.82.1:46563)
0925 09:21:17.188437 (+ 2296us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.188443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.188465 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:21:17.188538 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.188543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.188800 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.189022 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:21:17.189753 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.191414 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.194073 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.195553 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.195709 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.195936 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.197155 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.197331 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2140,"thread_start_us":2009,"threads_started":1}
I20260925 09:21:17.198081 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.185517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46563 (local address 127.0.0.1:33222)
0925 09:21:17.187800 (+ 2283us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:17.187817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:17.187938 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:17.189022 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:17.189027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:17.189057 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:21:17.189502 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.189515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.191595 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.191603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.193879 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.193894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.196102 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.196109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.196292 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.196337 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 09:21:17.196552 (+ 215us) client_negotiation.cc:240] Negotiation successful
0925 09:21:17.196853 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2165,"spinlock_wait_cycles":16768,"thread_start_us":967,"threads_started":1}
I20260925 09:21:17.201260 14167 tablet_service.cc:1467] Processing CreateTablet for tablet cfe8b5cfebb54511adc0e13af569d479 (DEFAULT_TABLE table=test-table [id=076de45172e74bda8bc640995139cd7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:21:17.201342 14033 tablet_service.cc:1467] Processing CreateTablet for tablet cfe8b5cfebb54511adc0e13af569d479 (DEFAULT_TABLE table=test-table [id=076de45172e74bda8bc640995139cd7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:21:17.201771 13899 tablet_service.cc:1467] Processing CreateTablet for tablet cfe8b5cfebb54511adc0e13af569d479 (DEFAULT_TABLE table=test-table [id=076de45172e74bda8bc640995139cd7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:21:17.202804 14033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe8b5cfebb54511adc0e13af569d479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:21:17.202827 14167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe8b5cfebb54511adc0e13af569d479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:21:17.203127 13899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe8b5cfebb54511adc0e13af569d479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:21:17.217142 14251 tablet_bootstrap.cc:492] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Bootstrap starting.
I20260925 09:21:17.220338 14252 tablet_bootstrap.cc:492] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Bootstrap starting.
I20260925 09:21:17.221261 14253 tablet_bootstrap.cc:492] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Bootstrap starting.
I20260925 09:21:17.222321 14251 tablet_bootstrap.cc:654] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Neither blocks nor log segments found. Creating new log.
I20260925 09:21:17.223841 14251 log.cc:824] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:21:17.224361 14253 tablet_bootstrap.cc:654] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Neither blocks nor log segments found. Creating new log.
I20260925 09:21:17.225359 14253 log.cc:824] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Log is configured to *not* fsync() on all Append() calls
I20260925 09:21:17.225562 14252 tablet_bootstrap.cc:654] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Neither blocks nor log segments found. Creating new log.
I20260925 09:21:17.227173 14252 log.cc:824] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Log is configured to *not* fsync() on all Append() calls
I20260925 09:21:17.228749 14253 tablet_bootstrap.cc:492] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: No bootstrap required, opened a new log
I20260925 09:21:17.228749 14251 tablet_bootstrap.cc:492] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: No bootstrap required, opened a new log
I20260925 09:21:17.229079 14251 ts_tablet_manager.cc:1397] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260925 09:21:17.229079 14253 ts_tablet_manager.cc:1397] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260925 09:21:17.230815 14252 tablet_bootstrap.cc:492] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: No bootstrap required, opened a new log
I20260925 09:21:17.231142 14252 ts_tablet_manager.cc:1397] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260925 09:21:17.240114 14252 raft_consensus.cc:348] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.240712 14252 raft_consensus.cc:374] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:21:17.240830 14252 raft_consensus.cc:729] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f62b9b8c17b64cb495adb831d9156f69, State: Initialized, Role: FOLLOWER
I20260925 09:21:17.240628 14253 raft_consensus.cc:348] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.240628 14251 raft_consensus.cc:348] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.241154 14251 raft_consensus.cc:374] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:21:17.241154 14253 raft_consensus.cc:374] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:21:17.241262 14251 raft_consensus.cc:729] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c7a6bb15bf4b908972c90d535085e2, State: Initialized, Role: FOLLOWER
I20260925 09:21:17.241262 14253 raft_consensus.cc:729] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae124341d35442bf8cab9a57b5ba529c, State: Initialized, Role: FOLLOWER
I20260925 09:21:17.241675 14252 consensus_queue.cc:260] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.241957 14253 consensus_queue.cc:260] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.241950 14251 consensus_queue.cc:260] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.245074 14232 heartbeater.cc:503] Master 127.3.82.62:43155 was elected leader, sending a full tablet report...
I20260925 09:21:17.245134 14251 ts_tablet_manager.cc:1428] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260925 09:21:17.246246 14253 ts_tablet_manager.cc:1428] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260925 09:21:17.246248 14252 ts_tablet_manager.cc:1428] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260925 09:21:17.246922 14098 heartbeater.cc:503] Master 127.3.82.62:43155 was elected leader, sending a full tablet report...
W20260925 09:21:17.315440 14099 tablet.cc:2369] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:21:17.353377 14233 tablet.cc:2369] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:21:17.447345 14259 raft_consensus.cc:482] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:21:17.447778 14259 raft_consensus.cc:504] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.449715 14259 leader_election.cc:288] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f62b9b8c17b64cb495adb831d9156f69 (127.3.82.1:46563), 25c7a6bb15bf4b908972c90d535085e2 (127.3.82.2:45337)
I20260925 09:21:17.458863 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.450919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:55763 (local address 127.3.82.1:46563)
0925 09:21:17.451128 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.451134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.451161 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:21:17.451238 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.451241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.451298 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.451377 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:21:17.451847 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.452861 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.455644 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.457240 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.457505 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.457640 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.458427 (+ 787us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.458581 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:21:17.458868 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.450449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:39191 (local address 127.3.82.2:45337)
0925 09:21:17.450668 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.450674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.451477 (+ 803us) server_negotiation.cc:407] Connection header received
0925 09:21:17.451687 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.451691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.451784 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.451886 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:21:17.452481 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.453348 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.455967 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.457416 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.457579 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.457690 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.458380 (+ 690us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.458546 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:21:17.458974 14235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.450189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:46563 (local address 127.3.82.3:55763)
0925 09:21:17.450494 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:17.450532 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:17.450704 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:17.451438 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:17.451445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:17.451473 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:21:17.451691 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.451702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.453045 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.453049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.455387 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.455408 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.457610 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.457626 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.458068 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.458169 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 09:21:17.458315 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 09:21:17.458541 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":137}
I20260925 09:21:17.459553 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.450361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:45337 (local address 127.3.82.3:39191)
0925 09:21:17.451321 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0925 09:21:17.451336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:21:17.451550 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:21:17.451952 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:21:17.451956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:21:17.451978 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:21:17.452294 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.452302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.453508 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.453519 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.455671 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:21:17.455681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.457797 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.457805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:21:17.458098 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.458170 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 09:21:17.458320 (+ 150us) client_negotiation.cc:240] Negotiation successful
0925 09:21:17.458541 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"mutex_wait_us":75,"spinlock_wait_cycles":25472,"thread_start_us":564,"threads_started":1}
I20260925 09:21:17.460284 13919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe8b5cfebb54511adc0e13af569d479" candidate_uuid: "ae124341d35442bf8cab9a57b5ba529c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62b9b8c17b64cb495adb831d9156f69" is_pre_election: true
I20260925 09:21:17.460865 14053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe8b5cfebb54511adc0e13af569d479" candidate_uuid: "ae124341d35442bf8cab9a57b5ba529c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c7a6bb15bf4b908972c90d535085e2" is_pre_election: true
I20260925 09:21:17.461038 13919 raft_consensus.cc:2454] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae124341d35442bf8cab9a57b5ba529c in term 0.
I20260925 09:21:17.461484 14053 raft_consensus.cc:2454] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae124341d35442bf8cab9a57b5ba529c in term 0.
I20260925 09:21:17.462049 14120 leader_election.cc:302] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae124341d35442bf8cab9a57b5ba529c, f62b9b8c17b64cb495adb831d9156f69; no voters:
I20260925 09:21:17.462595 14259 raft_consensus.cc:2781] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:21:17.462764 14259 raft_consensus.cc:482] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:21:17.462885 14259 raft_consensus.cc:3037] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:21:17.465435 14259 raft_consensus.cc:504] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
I20260925 09:21:17.466351 14259 leader_election.cc:288] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [CANDIDATE]: Term 1 election: Requested vote from peers f62b9b8c17b64cb495adb831d9156f69 (127.3.82.1:46563), 25c7a6bb15bf4b908972c90d535085e2 (127.3.82.2:45337)
I20260925 09:21:17.467031 13919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe8b5cfebb54511adc0e13af569d479" candidate_uuid: "ae124341d35442bf8cab9a57b5ba529c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f62b9b8c17b64cb495adb831d9156f69"
I20260925 09:21:17.467119 14053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe8b5cfebb54511adc0e13af569d479" candidate_uuid: "ae124341d35442bf8cab9a57b5ba529c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c7a6bb15bf4b908972c90d535085e2"
I20260925 09:21:17.467290 13919 raft_consensus.cc:3037] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:21:17.467437 14053 raft_consensus.cc:3037] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:21:17.470024 14053 raft_consensus.cc:2454] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae124341d35442bf8cab9a57b5ba529c in term 1.
I20260925 09:21:17.470039 13919 raft_consensus.cc:2454] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae124341d35442bf8cab9a57b5ba529c in term 1.
I20260925 09:21:17.470633 14123 leader_election.cc:302] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c7a6bb15bf4b908972c90d535085e2, ae124341d35442bf8cab9a57b5ba529c; no voters:
I20260925 09:21:17.471074 14259 raft_consensus.cc:2781] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:21:17.471635 14259 raft_consensus.cc:686] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 1 LEADER]: Becoming Leader. State: Replica: ae124341d35442bf8cab9a57b5ba529c, State: Running, Role: LEADER
I20260925 09:21:17.472190 14259 consensus_queue.cc:237] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } }
W20260925 09:21:17.477310 13965 tablet.cc:2369] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:21:17.477802 13774 catalog_manager.cc:5495] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c reported cstate change: term changed from 0 to 1, leader changed from <none> to ae124341d35442bf8cab9a57b5ba529c (127.3.82.3). New cstate: current_term: 1 leader_uuid: "ae124341d35442bf8cab9a57b5ba529c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae124341d35442bf8cab9a57b5ba529c" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 33655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 } health_report { overall_health: UNKNOWN } } }
I20260925 09:21:17.501437 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.494931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.3.82.3:33655)
0925 09:21:17.495169 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.495178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.495326 (+ 148us) server_negotiation.cc:407] Connection header received
0925 09:21:17.495569 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.495574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.495667 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.495925 (+ 258us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:21:17.496312 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.497117 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.498564 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.498988 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.499164 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.499748 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:21:17.500671 (+ 923us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:21:17.500836 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.500974 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.501102 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:21:17.513365 13919 raft_consensus.cc:1261] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 1 FOLLOWER]: Refusing update from remote peer ae124341d35442bf8cab9a57b5ba529c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:21:17.513365 14053 raft_consensus.cc:1261] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 1 FOLLOWER]: Refusing update from remote peer ae124341d35442bf8cab9a57b5ba529c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:21:17.514556 14259 consensus_queue.cc:1035] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f62b9b8c17b64cb495adb831d9156f69" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:21:17.514921 14261 consensus_queue.cc:1035] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c7a6bb15bf4b908972c90d535085e2" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:21:17.528106 14264 mvcc.cc:205] Tried to move back new op lower bound from 7333183805481549824 to 7333183805329899520. Current Snapshot: MvccSnapshot[applied={T|T < 7333183805481549824}]
I20260925 09:21:17.529893 14265 mvcc.cc:205] Tried to move back new op lower bound from 7333183805481549824 to 7333183805329899520. Current Snapshot: MvccSnapshot[applied={T|T < 7333183805481549824}]
I20260925 09:21:17.544800 13774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:21:17.545112 13774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:21:17.549124 13774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2177e86c46b643f8980cd55abf33fa35: Sending DeleteTablet for 3 replicas of tablet cfe8b5cfebb54511adc0e13af569d479
I20260925 09:21:17.550668 13899 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe8b5cfebb54511adc0e13af569d479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:21:17 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:33222
I20260925 09:21:17.550674 14033 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe8b5cfebb54511adc0e13af569d479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:21:17 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:46710
I20260925 09:21:17.551775 14273 tablet_replica.cc:330] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: stopping tablet replica
I20260925 09:21:17.551811 14167 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe8b5cfebb54511adc0e13af569d479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:21:17 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:41548
I20260925 09:21:17.555599 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13836
I20260925 09:21:17.556717 14273 raft_consensus.cc:2229] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:21:17.557425 14273 raft_consensus.cc:2258] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:21:17.574358 14274 tablet_replica.cc:330] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: stopping tablet replica
I20260925 09:21:17.574982 14274 raft_consensus.cc:2229] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:21:17.575661 14274 raft_consensus.cc:2258] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:21:17.576284 14273 ts_tablet_manager.cc:1905] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:21:17.578383 14274 ts_tablet_manager.cc:1905] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:21:17.580789 14275 tablet_replica.cc:330] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: stopping tablet replica
I20260925 09:21:17.581284 14275 raft_consensus.cc:2229] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:21:17.582010 14275 raft_consensus.cc:2258] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:21:17.586539 14275 ts_tablet_manager.cc:1905] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:21:17.587456 14273 ts_tablet_manager.cc:1918] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:21:17.587699 14273 log.cc:1196] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/cfe8b5cfebb54511adc0e13af569d479
I20260925 09:21:17.588323 14273 ts_tablet_manager.cc:1939] T cfe8b5cfebb54511adc0e13af569d479 P f62b9b8c17b64cb495adb831d9156f69: Deleting consensus metadata
I20260925 09:21:17.590430 13759 catalog_manager.cc:4841] TS f62b9b8c17b64cb495adb831d9156f69 (127.3.82.1:46563): tablet cfe8b5cfebb54511adc0e13af569d479 (table test-table [id=076de45172e74bda8bc640995139cd7c]) successfully deleted
I20260925 09:21:17.593796 14275 ts_tablet_manager.cc:1918] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:21:17.594053 14275 log.cc:1196] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/cfe8b5cfebb54511adc0e13af569d479
I20260925 09:21:17.594739 14275 ts_tablet_manager.cc:1939] T cfe8b5cfebb54511adc0e13af569d479 P ae124341d35442bf8cab9a57b5ba529c: Deleting consensus metadata
Sep 25 09:21:17 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:21:17.596802 13759 catalog_manager.cc:4841] TS ae124341d35442bf8cab9a57b5ba529c (127.3.82.3:33655): tablet cfe8b5cfebb54511adc0e13af569d479 (table test-table [id=076de45172e74bda8bc640995139cd7c]) successfully deleted
I20260925 09:21:17.601177 14274 ts_tablet_manager.cc:1918] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:21:17.601455 14274 log.cc:1196] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/cfe8b5cfebb54511adc0e13af569d479
I20260925 09:21:17.602160 14274 ts_tablet_manager.cc:1939] T cfe8b5cfebb54511adc0e13af569d479 P 25c7a6bb15bf4b908972c90d535085e2: Deleting consensus metadata
I20260925 09:21:17.604154 13762 catalog_manager.cc:4841] TS 25c7a6bb15bf4b908972c90d535085e2 (127.3.82.2:45337): tablet cfe8b5cfebb54511adc0e13af569d479 (table test-table [id=076de45172e74bda8bc640995139cd7c]) successfully deleted
I20260925 09:21:17.605759 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.557084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.3.82.1:46563)
0925 09:21:17.557265 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.557273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.574695 (+ 17422us) server_negotiation.cc:407] Connection header received
0925 09:21:17.575240 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.575262 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.575361 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.575439 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:17.577329 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.578508 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.582451 (+ 3943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.583147 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.583263 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.597608 (+ 14345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:17.597630 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:17.597651 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:17.597699 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:17.602149 (+ 4450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.603250 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.603261 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.603265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.603448 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.603934 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.603939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.603942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.604472 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:17.604782 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.605114 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.605356 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 09:21:17.606837 13959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:17.699919 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13836
I20260925 09:21:17.723451 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13970
Sep 25 09:21:17 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:21:17.742677 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.724441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.3.82.2:45337)
0925 09:21:17.724620 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.724624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.724642 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:21:17.724743 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.724748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.724837 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.724925 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:17.726644 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.727459 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.729429 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.730062 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.730179 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.735607 (+ 5428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:17.735625 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:17.735642 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:17.735696 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:17.739427 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.740300 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.740308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.740310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.740449 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.740855 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.740865 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.740867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.741280 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:17.741586 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.742085 (+ 499us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.742337 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:21:17.743429 14093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:17.856953 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13970
I20260925 09:21:17.875558 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 14104
Sep 25 09:21:17 dist-test-slave-4wm5 krb5kdc[13730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:21:17.892686 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:21:17.876451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.3.82.3:33655)
0925 09:21:17.876645 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 09:21:17.876652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:21:17.876694 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:21:17.876917 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:21:17.876924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:21:17.877007 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:21:17.877135 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:21:17.878525 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.879384 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.880720 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:21:17.881202 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:21:17.881366 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:21:17.885956 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:21:17.885974 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:21:17.885991 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:21:17.886033 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:21:17.889425 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.890439 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.890448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.890452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.890627 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:21:17.891026 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:21:17.891033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:21:17.891040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:21:17.891386 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:21:17.891740 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0925 09:21:17.892029 (+ 289us) server_negotiation.cc:299] Negotiation successful
0925 09:21:17.892276 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:21:17.893392 14227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:18.023304 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 14104
I20260925 09:21:18.041080 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 13743
I20260925 09:21:18.042542 13805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:21:18.158118 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 13743
2026-09-25T09:21:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4766 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:21:18 dist-test-slave-4wm5 krb5kdc[14283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:21:18 dist-test-slave-4wm5 krb5kdc[14283](info): set up 2 sockets
Sep 25 09:21:18 dist-test-slave-4wm5 krb5kdc[14283](info): commencing operation
krb5kdc: starting...
W20260925 09:21:19.351504 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:21:19 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:21:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:21:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/rangeradmin_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/rangeradmin_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/rangerlookup_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/rangerlookup_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
I20260925 09:21:19.723225 3400 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0mddRs/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/postgres -l logfile start
2026-09-25 09:21:22.683 UTC [14313] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 09:21:22.683 UTC [14313] LOG: listening on IPv4 address "127.3.82.20", port 47683
2026-09-25 09:21:22.687 UTC [14313] LOG: listening on Unix socket "/tmp/.s.PGSQL.47683"
2026-09-25 09:21:22.712 UTC [14316] LOG: database system was shut down at 2026-09-25 09:21:20 UTC
2026-09-25 09:21:22.716 UTC [14313] LOG: database system is ready to accept connections
I20260925 09:21:23.794646 3400 mini_postgres.cc:96] Postgres bound to 47683
2026-09-25 09:21:23.814 UTC [14324] FATAL: database "slave" does not exist
127.3.82.20:47683 - accepting connections
I20260925 09:21:23.815585 3400 mini_ranger.cc:161] Starting Ranger...
I20260925 09:21:23.845902 3400 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 09:21:24.454922 3400 mini_ranger.cc:87] Created ranger Postgres database
I20260925 09:21:24.455052 3400 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-admin
2026-09-25 09:21:24,757 [I] DB FLAVOR :POSTGRES
2026-09-25 09:21:24,758 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:21:24,758 [I] Checking connection..
2026-09-25 09:21:24,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 09:21:25,027 [I] Checking connection passed.
2026-09-25 09:21:25,027 [I] --------- Verifying version history table ---------
2026-09-25 09:21:25,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:21:25,301 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:21:25,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:21:25,578 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:21:25,578 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 09:21:25,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 09:21:25.855 UTC [14388] WARNING: there is no transaction in progress
2026-09-25 09:21:25,862 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 09:21:25,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:21:26,140 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 09:21:26,141 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:21:26,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 09:21:26,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 09:21:26,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4wm5.c.gcp-upstream.internal','N') ;"
2026-09-25 09:21:26,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:21:27,224 [I] Table x_portal_user does not exist in database ranger
2026-09-25 09:21:27,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:21:27,496 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 09:21:27,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:21:27,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:21:28,007 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 09:21:28,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 09:21:29.150 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.169 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.188 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.207 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.225 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.246 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.320 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.328 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.346 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.358 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.370 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.387 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.399 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.407 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.417 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:29.425 UTC [14496] WARNING: there is no transaction in progress
2026-09-25 09:21:30,286 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 09:21:30,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 09:21:30,540 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 09:21:30,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:21:30,791 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 09:21:30,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:21:31,070 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 09:21:31,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:21:31,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:21:31,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4wm5.c.gcp-upstream.internal';"
2026-09-25 09:21:31,821 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 09:21:31,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 09:21:32,096 [I] DB_PATCHES have already been applied
I20260925 09:21:32.102355 3400 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-admin:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0mddRs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 09:21:32.102460 3400 mini_ranger.cc:193] Using host: 127.3.82.20
I20260925 09:21:32.119392 3400 mini_ranger.cc:227] Ranger admin URL: http://127.3.82.20:34103
Sep 25, 2026 9:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 9:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 9:21:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 9:21:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 9:21:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 09:21:33.309255 3400 mini_ranger.cc:160] Time spent starting Ranger: real 9.494s user 0.004s sys 0.042s
Sep 25, 2026 9:21:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 9:21:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 25, 2026 9:21:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 9:21:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 9:21:55 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 09:21:55.731244 3400 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/rangerkms_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/rangerkms_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.82.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.82.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 09:21:55 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328115, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 09:21:55.916074 3400 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 09:21:55.939121 3400 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 09:21:57.056569 3400 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 09:21:57.056752 3400 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms
I20260925 09:21:57.056887 3400 mini_ranger_kms.cc:227] Using postgres at 127.3.82.20:47683
2026-09-25 09:21:57,490 [I] DB FLAVOR :POSTGRES
2026-09-25 09:21:57,491 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:21:57,491 [I] Checking connection
2026-09-25 09:21:57,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 09:21:57,739 [I] connection success
2026-09-25 09:21:57,739 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 09:21:57,739 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 09:21:57,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 09:21:57,997 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 09:21:57,997 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:21:57,997 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 09:21:57,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:47683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 09:21:58,300 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 09:21:58.852039 3400 mini_ranger_kms.cc:310] Created kms service
I20260925 09:21:59.101105 3400 mini_ranger_kms.cc:326] Created kudu user
I20260925 09:21:59.158439 3400 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 09:21:59 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328115, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:21:59.605785 3400 mini_ranger_kms.cc:384] Added ranger policy
I20260925 09:21:59.606159 3400 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms:/tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 09:21:59.606228 3400 mini_ranger_kms.cc:240] Using host: 127.3.82.20
I20260925 09:21:59.624850 3400 mini_ranger_kms.cc:276] Ranger KMS PID: 14677
I20260925 09:21:59.625069 3400 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.82.20:55253
Sep 25, 2026 9:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 9:22:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 9:22:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 9:22:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 09:22:01.972908 3400 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.057s user 0.137s sys 0.318s
I20260925 09:22:01.973186 3400 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 09:22:01.973403 3400 mini_ranger_kms.cc:401] 127.3.82.20:55253/kms/v1/keys
Sep 25 09:22:02 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328122, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.82.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 09:22:02 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328122, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.82.20@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
Sep 25 09:22:02 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328122, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.82.20@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
Sep 25 09:22:05 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:22:05.490350 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45873
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:45873
--ranger_config_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55253/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:22:05.850266 14727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:22:05.850643 14727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:22:05.850824 14727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:22:05.858680 14727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:22:05.858872 14727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:22:05.858989 14727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:22:05.859089 14727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:22:05.859179 14727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:22:05.870507 14727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.82.20:55253/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:45873
--ranger_config_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:45873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:22:05.871946 14727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:22:05.873335 14727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:22:05.886508 14735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:05.886535 14733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:05.886664 14732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:22:05.886765 14727 server_base.cc:1034] running on GCE node
I20260925 09:22:05.888365 14727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:22:05.890019 14727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:22:05.891289 14727 hybrid_clock.cc:648] HybridClock initialized: now 1790328125891245 us; error 39 us; skew 500 ppm
I20260925 09:22:05.891682 14727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:22:05 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:22:05.895258 14727 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:22:05.896914 14727 webserver.cc:460] Webserver started at http://127.3.82.62:34145/ using document root <none> and password file <none>
I20260925 09:22:05.897645 14727 fs_manager.cc:359] Metadata directory not provided
I20260925 09:22:05.897784 14727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:22:05.898200 14727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:22:05 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:05.965237 14727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "5c9a9df26c6943fca44f3d979dce4be0"
format_stamp: "Formatted at 2026-09-25 09:22:05 on dist-test-slave-4wm5"
server_key: "953ef15ef6d8a38fbd5764e2314a7e99"
server_key_iv: "89216ba6275c8c57d91ade5c390507db"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:05.966418 14727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "5c9a9df26c6943fca44f3d979dce4be0"
format_stamp: "Formatted at 2026-09-25 09:22:05 on dist-test-slave-4wm5"
server_key: "953ef15ef6d8a38fbd5764e2314a7e99"
server_key_iv: "89216ba6275c8c57d91ade5c390507db"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:05.973980 14727 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260925 09:22:05.996361 14742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:05.998324 14727 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 09:22:05.998540 14727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "5c9a9df26c6943fca44f3d979dce4be0"
format_stamp: "Formatted at 2026-09-25 09:22:05 on dist-test-slave-4wm5"
server_key: "953ef15ef6d8a38fbd5764e2314a7e99"
server_key_iv: "89216ba6275c8c57d91ade5c390507db"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:05.998808 14727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:22:06.017959 14727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:22:06.023384 14727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:22:06.023728 14727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:22:06.051903 14727 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:45873
I20260925 09:22:06.051960 14793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:45873 every 8 connection(s)
I20260925 09:22:06.053423 14727 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:22:06.053870 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 14727
I20260925 09:22:06.054095 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:22:06.055092 3400 external_mini_cluster.cc:1428] Setting key bf14db74dcf289a5977d4ec81b6054b3
I20260925 09:22:06.065140 14794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:22:06 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:22:06.084538 14794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0: Bootstrap starting.
I20260925 09:22:06.088809 14794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0: Neither blocks nor log segments found. Creating new log.
I20260925 09:22:06.090212 14794 log.cc:824] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:22:06.092466 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:06.057466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54638 (local address 127.3.82.62:45873)
0925 09:22:06.069879 (+ 12413us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:06.069903 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:06.070587 (+ 684us) server_negotiation.cc:407] Connection header received
0925 09:22:06.073387 (+ 2800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:06.073415 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:06.074199 (+ 784us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:06.074695 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:06.076183 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:06.077614 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:06.078974 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:06.080115 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:06.080340 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:06.084464 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:06.084473 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:06.084490 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:06.084536 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:06.087791 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:06.088591 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:06.088598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:06.088605 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:06.088771 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:06.089198 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:06.089203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:06.089208 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:06.089784 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:06.090126 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:06.090640 (+ 514us) server_negotiation.cc:299] Negotiation successful
0925 09:22:06.091086 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12039,"thread_start_us":11794,"threads_started":1}
I20260925 09:22:06.094054 14794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0: No bootstrap required, opened a new log
I20260925 09:22:06.104904 14794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a9df26c6943fca44f3d979dce4be0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45873 } }
I20260925 09:22:06.105505 14794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:22:06.105625 14794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9a9df26c6943fca44f3d979dce4be0, State: Initialized, Role: FOLLOWER
I20260925 09:22:06.106416 14794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a9df26c6943fca44f3d979dce4be0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45873 } }
I20260925 09:22:06.106804 14794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:22:06.106993 14794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:22:06.107173 14794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:22:06.110416 14794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a9df26c6943fca44f3d979dce4be0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45873 } }
I20260925 09:22:06.111089 14794 leader_election.cc:302] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c9a9df26c6943fca44f3d979dce4be0; no voters:
I20260925 09:22:06.111812 14794 leader_election.cc:288] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:22:06.112260 14799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:22:06.114833 14794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:22:06.118206 14799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [term 1 LEADER]: Becoming Leader. State: Replica: 5c9a9df26c6943fca44f3d979dce4be0, State: Running, Role: LEADER
I20260925 09:22:06.119066 14799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a9df26c6943fca44f3d979dce4be0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45873 } }
I20260925 09:22:06.126567 14801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c9a9df26c6943fca44f3d979dce4be0. Latest consensus state: current_term: 1 leader_uuid: "5c9a9df26c6943fca44f3d979dce4be0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a9df26c6943fca44f3d979dce4be0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45873 } } }
I20260925 09:22:06.126997 14801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:22:06.127642 14800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c9a9df26c6943fca44f3d979dce4be0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a9df26c6943fca44f3d979dce4be0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 45873 } } }
I20260925 09:22:06.127934 14800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:22:06.130584 14809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:22:06.136760 14809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:22:06.137585 14794 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 09:22:06.165709 14809 catalog_manager.cc:1347] Generated new cluster ID: 9f3d0213eaed422e899f19ee85526054
I20260925 09:22:06.166060 14809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:22:06.195756 14809 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:22:06.198798 14809 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:22:06.225574 14809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0: Generated new TSK 0
I20260925 09:22:06.226925 14809 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:22:07.071058 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55253/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45873
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:22:07 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 09:22:07.564337 14839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:22:07.564857 14839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:22:07.565026 14839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:22:07.574209 14839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:22:07.574395 14839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:22:07.574594 14839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:22:07.585961 14839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.82.20:55253/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:22:07.587430 14839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:22:07.588812 14839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:22:07.617481 14849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:07.619074 14850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:07.627585 14852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:22:07.631029 14839 server_base.cc:1034] running on GCE node
I20260925 09:22:07.631954 14839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:22:07.634917 14839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:22:07.636283 14839 hybrid_clock.cc:648] HybridClock initialized: now 1790328127636148 us; error 108 us; skew 500 ppm
I20260925 09:22:07.636754 14839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:22:07 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:22:07.641286 14839 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:22:07.643913 14839 webserver.cc:460] Webserver started at http://127.3.82.1:35135/ using document root <none> and password file <none>
I20260925 09:22:07.644901 14839 fs_manager.cc:359] Metadata directory not provided
I20260925 09:22:07.645133 14839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:22:07.645571 14839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:22:07 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:07.677146 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "3c5ba3a2535b466d9dfa172e3076020f"
format_stamp: "Formatted at 2026-09-25 09:22:07 on dist-test-slave-4wm5"
server_key: "5ed14cd0b2e95a48ade7b3f43cd18c1e"
server_key_iv: "d571ef21235830cd6389e63059723daa"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:07.678819 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "3c5ba3a2535b466d9dfa172e3076020f"
format_stamp: "Formatted at 2026-09-25 09:22:07 on dist-test-slave-4wm5"
server_key: "5ed14cd0b2e95a48ade7b3f43cd18c1e"
server_key_iv: "d571ef21235830cd6389e63059723daa"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:07.701807 14839 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.011s sys 0.008s
I20260925 09:22:07.731452 14859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:07.740072 14839 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.008s sys 0.000s
I20260925 09:22:07.740360 14839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "3c5ba3a2535b466d9dfa172e3076020f"
format_stamp: "Formatted at 2026-09-25 09:22:07 on dist-test-slave-4wm5"
server_key: "5ed14cd0b2e95a48ade7b3f43cd18c1e"
server_key_iv: "d571ef21235830cd6389e63059723daa"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:07.740654 14839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:22:07.806087 14839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:22:07.812057 14839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:22:07.812754 14839 kserver.cc:163] Server-wide thread pool size limit: 3276
Sep 25 09:22:07 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:07.824977 14839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:22:07.833575 14839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:22:07.833808 14839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 09:22:07.834095 14839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:22:07.834236 14839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:07.930068 14839 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:43949
I20260925 09:22:07.930121 14971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:43949 every 8 connection(s)
I20260925 09:22:07.931874 14839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:22:07.939066 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 14839
I20260925 09:22:07.939286 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:22:07.940083 3400 external_mini_cluster.cc:1428] Setting key 74fb66fa98c3706287cd99de16fba634
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Sep 25 09:22:07 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:22:07.988176 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:07.935012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:51365 (local address 127.3.82.62:45873)
0925 09:22:07.956994 (+ 21982us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:07.957005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:07.960792 (+ 3787us) server_negotiation.cc:407] Connection header received
0925 09:22:07.960878 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:07.960884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:07.960999 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:07.961120 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:07.963553 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:07.964713 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:07.971402 (+ 6689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:07.972051 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:07.972193 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:07.979322 (+ 7129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:07.979329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:07.979334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:07.979380 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:07.983112 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:07.984355 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:07.984365 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:07.984371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:07.984552 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:07.985661 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:07.985669 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:07.985673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:07.986159 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:07.986348 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:07.987380 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0925 09:22:07.987666 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21768,"thread_start_us":21606,"threads_started":1}
I20260925 09:22:07.989254 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:07.935879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45873 (local address 127.3.82.1:51365)
0925 09:22:07.957652 (+ 21773us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:07.957710 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:07.960235 (+ 2525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:07.961671 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:07.961685 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:07.962390 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:22:07.963302 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:07.963332 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:07.964939 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:07.964946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:07.971106 (+ 6160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:07.971135 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:07.972553 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:07.972562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:07.972886 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:07.973750 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:22:07.973781 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:22:07.978967 (+ 5186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:22:07.983382 (+ 4415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:22:07.983393 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:22:07.983417 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:22:07.984123 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:22:07.984720 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:22:07.984725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:22:07.984728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:22:07.985478 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:22:07.986343 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:22:07.986353 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:22:07.986862 (+ 509us) client_negotiation.cc:769] Sending connection context
0925 09:22:07.987209 (+ 347us) client_negotiation.cc:240] Negotiation successful
0925 09:22:07.987826 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21438,"thread_start_us":21179,"threads_started":1}
I20260925 09:22:07.991182 14972 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45873
I20260925 09:22:07.991539 14972 heartbeater.cc:464] Registering TS with master...
I20260925 09:22:07.992475 14972 heartbeater.cc:511] Master 127.3.82.62:45873 requested a full tablet report, sending...
I20260925 09:22:07.995209 14759 ts_manager.cc:194] Registered new tserver with Master: 3c5ba3a2535b466d9dfa172e3076020f (127.3.82.1:43949)
I20260925 09:22:07.997344 14759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:51365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:22:08.013216 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55253/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45873
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:22:08 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
W20260925 09:22:08.460289 14978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:22:08.460645 14978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:22:08.460827 14978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:22:08.472739 14978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:22:08.473426 14978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:22:08.473671 14978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:22:08.487906 14978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.82.20:55253/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:22:08.489650 14978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:22:08.491325 14978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:22:08.506599 14992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:08.506845 14993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:08.506896 14995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:22:08.508551 14978 server_base.cc:1034] running on GCE node
I20260925 09:22:08.509291 14978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:22:08.510571 14978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:22:08.511914 14978 hybrid_clock.cc:648] HybridClock initialized: now 1790328128511822 us; error 80 us; skew 500 ppm
I20260925 09:22:08.512310 14978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:22:08 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328128, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:22:08.516004 14978 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:22:08.517621 14978 webserver.cc:460] Webserver started at http://127.3.82.2:46275/ using document root <none> and password file <none>
I20260925 09:22:08.518405 14978 fs_manager.cc:359] Metadata directory not provided
I20260925 09:22:08.518559 14978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:22:08.518896 14978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:22:08 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328128, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:08.535738 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "d53bcb857b4e42a6940069f24c55dd51"
format_stamp: "Formatted at 2026-09-25 09:22:08 on dist-test-slave-4wm5"
server_key: "1c97692bb5022523b3b99b83f4bb5049"
server_key_iv: "84e3abef10db94dbe1c772c35890b1e5"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:08.536744 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "d53bcb857b4e42a6940069f24c55dd51"
format_stamp: "Formatted at 2026-09-25 09:22:08 on dist-test-slave-4wm5"
server_key: "1c97692bb5022523b3b99b83f4bb5049"
server_key_iv: "84e3abef10db94dbe1c772c35890b1e5"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:08.543015 14978 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:22:08.560702 15002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:08.562713 14978 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:22:08.562917 14978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "d53bcb857b4e42a6940069f24c55dd51"
format_stamp: "Formatted at 2026-09-25 09:22:08 on dist-test-slave-4wm5"
server_key: "1c97692bb5022523b3b99b83f4bb5049"
server_key_iv: "84e3abef10db94dbe1c772c35890b1e5"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:08.563210 14978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:22:08.593436 14978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:22:08.598963 14978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:22:08.599330 14978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:22:08.600628 14978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:22:08.602552 14978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:22:08.602658 14978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:08.602805 14978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:22:08.602872 14978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:08.643100 14978 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:33837
I20260925 09:22:08.643134 15114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:33837 every 8 connection(s)
I20260925 09:22:08.644945 14978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:22:08.652822 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 14978
I20260925 09:22:08.653033 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:22:08.653570 3400 external_mini_cluster.cc:1428] Setting key 36bd43019f280f099993b1a9de917a63
Sep 25 09:22:08 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328128, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:22:08.670249 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:08.648226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:39143 (local address 127.3.82.62:45873)
0925 09:22:08.648674 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:08.648678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:08.649612 (+ 934us) server_negotiation.cc:407] Connection header received
0925 09:22:08.651559 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:08.651565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:08.651672 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:08.651795 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:08.653786 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:08.654859 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:08.657149 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:08.657587 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:08.657698 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:08.662551 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:08.662557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:08.662561 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:08.662597 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:08.666390 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:08.667299 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:08.667308 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:08.667314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:08.667493 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:08.667985 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:08.667993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:08.667999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:08.668418 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:08.668576 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:08.669439 (+ 863us) server_negotiation.cc:299] Negotiation successful
0925 09:22:08.669716 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":242,"threads_started":1}
I20260925 09:22:08.671135 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:08.648704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45873 (local address 127.3.82.2:39143)
0925 09:22:08.649449 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:08.649487 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:08.651250 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:08.652045 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:08.652054 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:08.652603 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:22:08.653513 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:08.653542 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:08.655029 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:08.655035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:08.656955 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:08.656972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:08.657888 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:08.657900 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:08.658276 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:08.659109 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:22:08.659147 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:22:08.662247 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:22:08.666581 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:22:08.666590 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:22:08.666606 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:22:08.667152 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:22:08.667620 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:22:08.667624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:22:08.667628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:22:08.667862 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:22:08.668553 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:22:08.668559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:22:08.668998 (+ 439us) client_negotiation.cc:769] Sending connection context
0925 09:22:08.669312 (+ 314us) client_negotiation.cc:240] Negotiation successful
0925 09:22:08.669818 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":305,"threads_started":1}
I20260925 09:22:08.672837 15115 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45873
I20260925 09:22:08.673162 15115 heartbeater.cc:464] Registering TS with master...
I20260925 09:22:08.673971 15115 heartbeater.cc:511] Master 127.3.82.62:45873 requested a full tablet report, sending...
I20260925 09:22:08.675748 14759 ts_manager.cc:194] Registered new tserver with Master: d53bcb857b4e42a6940069f24c55dd51 (127.3.82.2:33837)
I20260925 09:22:08.676589 14759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:39143
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:22:08.714190 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55253/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45873
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:22:09.002166 14972 heartbeater.cc:503] Master 127.3.82.62:45873 was elected leader, sending a full tablet report...
W20260925 09:22:09.078220 15121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:22:09.078569 15121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:22:09.078836 15121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:22:09.086634 15121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:22:09.086822 15121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:22:09.087005 15121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:22:09.097144 15121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.82.20:55253/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:45873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:22:09.098726 15121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:22:09.100138 15121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:22:09.114840 15126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:09.114895 15127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:09.114953 15129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:22:09.116312 15121 server_base.cc:1034] running on GCE node
I20260925 09:22:09.116997 15121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:22:09.118054 15121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:22:09.119314 15121 hybrid_clock.cc:648] HybridClock initialized: now 1790328129119266 us; error 48 us; skew 500 ppm
I20260925 09:22:09.119678 15121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:22:09 dist-test-slave-4wm5 krb5kdc[14283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328129, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:22:09.127868 15121 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:22:09.129566 15121 webserver.cc:460] Webserver started at http://127.3.82.3:42055/ using document root <none> and password file <none>
I20260925 09:22:09.130342 15121 fs_manager.cc:359] Metadata directory not provided
I20260925 09:22:09.130498 15121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:22:09.130811 15121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:22:09 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328129, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:09.148085 15121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "8d1d1d190a69423990badbf744e4914e"
format_stamp: "Formatted at 2026-09-25 09:22:09 on dist-test-slave-4wm5"
server_key: "952f18b5b26a0f70257c24f600de404a"
server_key_iv: "658602bb4cf7a2dd7b86f740adb9ccba"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:09.149406 15121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "8d1d1d190a69423990badbf744e4914e"
format_stamp: "Formatted at 2026-09-25 09:22:09 on dist-test-slave-4wm5"
server_key: "952f18b5b26a0f70257c24f600de404a"
server_key_iv: "658602bb4cf7a2dd7b86f740adb9ccba"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:09.156212 15121 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260925 09:22:09.171706 15136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:09.173743 15121 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:22:09.173991 15121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "8d1d1d190a69423990badbf744e4914e"
format_stamp: "Formatted at 2026-09-25 09:22:09 on dist-test-slave-4wm5"
server_key: "952f18b5b26a0f70257c24f600de404a"
server_key_iv: "658602bb4cf7a2dd7b86f740adb9ccba"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:09.174283 15121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:22:09.204394 15121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:22:09.209721 15121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:22:09.210086 15121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:22:09.211318 15121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:22:09.213068 15121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:22:09.213167 15121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:09.213302 15121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:22:09.213371 15121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:09.272534 15121 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:35065
I20260925 09:22:09.272636 15248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:35065 every 8 connection(s)
I20260925 09:22:09.273948 15121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:22:09.283273 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 15121
I20260925 09:22:09.283464 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:22:09.283900 3400 external_mini_cluster.cc:1428] Setting key bf05329f9840255a0f560edc2af46a60
Sep 25 09:22:09 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328129, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:22:09.299538 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.277474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:50017 (local address 127.3.82.62:45873)
0925 09:22:09.278049 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.278056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.279241 (+ 1185us) server_negotiation.cc:407] Connection header received
0925 09:22:09.281424 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.281430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.281537 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.281670 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:09.283826 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.284972 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.287333 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.287968 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.288100 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.292037 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:09.292043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:09.292047 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:09.292095 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:09.295429 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.296516 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.296521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.296523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.296684 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.297282 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.297286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.297289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.297671 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:09.297870 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.298862 (+ 992us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.299118 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":267,"threads_started":1}
I20260925 09:22:09.300758 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.278146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:45873 (local address 127.3.82.3:50017)
0925 09:22:09.279032 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:09.279104 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:09.281098 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:09.282046 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:09.282055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:09.282525 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:22:09.283356 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.283386 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.285302 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.285308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.286976 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.286995 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.288191 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.288197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.288417 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.289263 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:22:09.289287 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:22:09.291681 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:22:09.295629 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:22:09.295641 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:22:09.295663 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:22:09.296341 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:22:09.296816 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:22:09.296822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:22:09.296826 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:22:09.297124 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:22:09.297865 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:22:09.297873 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:22:09.298350 (+ 477us) client_negotiation.cc:769] Sending connection context
0925 09:22:09.298669 (+ 319us) client_negotiation.cc:240] Negotiation successful
0925 09:22:09.299228 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":378,"threads_started":1}
I20260925 09:22:09.302887 15249 heartbeater.cc:347] Connected to a master server at 127.3.82.62:45873
I20260925 09:22:09.303243 15249 heartbeater.cc:464] Registering TS with master...
I20260925 09:22:09.304119 15249 heartbeater.cc:511] Master 127.3.82.62:45873 requested a full tablet report, sending...
I20260925 09:22:09.306015 14759 ts_manager.cc:194] Registered new tserver with Master: 8d1d1d190a69423990badbf744e4914e (127.3.82.3:35065)
I20260925 09:22:09.306882 14759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:50017
I20260925 09:22:09.312849 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:22:09.336556 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.319898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.3.82.62:45873)
0925 09:22:09.320130 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.320140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.320587 (+ 447us) server_negotiation.cc:407] Connection header received
0925 09:22:09.320776 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.320780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.320874 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.320966 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:09.322978 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.323917 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.325902 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.326394 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.326509 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.329602 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:09.329609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:09.329612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:09.329661 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:09.333006 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.334002 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.334007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.334010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.334611 (+ 601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.335138 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.335142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.335144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.335486 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:09.335680 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.336123 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.336325 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:22:09.341377 14759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:22:09.349023 14759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:22:09.390156 15262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.363145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:35065 (local address 127.0.0.1:54518)
0925 09:22:09.370493 (+ 7348us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:09.380426 (+ 9933us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:09.380576 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:09.381549 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:09.381554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:09.381580 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:22:09.382110 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.382123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.384564 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.384570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.386918 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.386933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.389096 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.389102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.389313 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.389348 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 09:22:09.389479 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 09:22:09.389698 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7071,"mutex_wait_us":6,"thread_start_us":6898,"threads_started":1}
I20260925 09:22:09.390256 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.363955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54814 (local address 127.3.82.1:43949)
0925 09:22:09.380601 (+ 16646us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.380613 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.380642 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:22:09.380733 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.380740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.381103 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.381339 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:22:09.382299 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.383830 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.386925 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.388472 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.388578 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.388832 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.389655 (+ 823us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.389895 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16387,"thread_start_us":16170,"threads_started":1}
I20260925 09:22:09.391048 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.363280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:43949 (local address 127.0.0.1:54814)
0925 09:22:09.377977 (+ 14697us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:09.378049 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:09.378299 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:09.381585 (+ 3286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:09.381591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:09.381617 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:22:09.382096 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.382110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.384011 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.384020 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.386690 (+ 2670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.386715 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.389016 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.389022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.389250 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.389347 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 09:22:09.389480 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:22:09.390632 (+ 1152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14524,"mutex_wait_us":1149,"thread_start_us":14387,"threads_started":1}
I20260925 09:22:09.391659 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.364948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.3.82.2:33837)
0925 09:22:09.378324 (+ 13376us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.378336 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.379009 (+ 673us) server_negotiation.cc:407] Connection header received
0925 09:22:09.379117 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.379125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.379521 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.379724 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:22:09.380499 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.382100 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.388353 (+ 6253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.389867 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.389965 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.390160 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.391146 (+ 986us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.391366 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13225,"thread_start_us":13132,"threads_started":1}
I20260925 09:22:09.391916 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.363708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.3.82.3:35065)
0925 09:22:09.374919 (+ 11211us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.374933 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.380702 (+ 5769us) server_negotiation.cc:407] Connection header received
0925 09:22:09.380808 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.380815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.381064 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.381272 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:22:09.382335 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.384366 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.387105 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.388606 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.388732 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.388949 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.391050 (+ 2101us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.391450 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10987,"thread_start_us":10798,"threads_started":1}
I20260925 09:22:09.392585 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.364759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:33837 (local address 127.0.0.1:37310)
0925 09:22:09.378742 (+ 13983us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:09.378773 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:09.378887 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:09.379874 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:09.379882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:09.379910 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:22:09.380305 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.380319 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.383681 (+ 3362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.383686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.388143 (+ 4457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.388157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.390360 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.390367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.390535 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.390566 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:22:09.390670 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 09:22:09.390871 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13864,"mutex_wait_us":1631,"thread_start_us":13726,"threads_started":1}
I20260925 09:22:09.395933 14907 tablet_service.cc:1467] Processing CreateTablet for tablet a12d476c71544a6aa5fa39c225207ff6 (DEFAULT_TABLE table=test-table [id=34006516f6044f6abb9d8b568a221e58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:22:09.396292 15184 tablet_service.cc:1467] Processing CreateTablet for tablet a12d476c71544a6aa5fa39c225207ff6 (DEFAULT_TABLE table=test-table [id=34006516f6044f6abb9d8b568a221e58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:22:09.396288 15050 tablet_service.cc:1467] Processing CreateTablet for tablet a12d476c71544a6aa5fa39c225207ff6 (DEFAULT_TABLE table=test-table [id=34006516f6044f6abb9d8b568a221e58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:22:09.397722 15184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12d476c71544a6aa5fa39c225207ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:22:09.397722 15050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12d476c71544a6aa5fa39c225207ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:22:09.398061 14907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a12d476c71544a6aa5fa39c225207ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:22:09.423848 15270 tablet_bootstrap.cc:492] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Bootstrap starting.
I20260925 09:22:09.425500 15269 tablet_bootstrap.cc:492] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Bootstrap starting.
I20260925 09:22:09.426609 15268 tablet_bootstrap.cc:492] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Bootstrap starting.
I20260925 09:22:09.430500 15268 tablet_bootstrap.cc:654] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Neither blocks nor log segments found. Creating new log.
I20260925 09:22:09.430553 15270 tablet_bootstrap.cc:654] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Neither blocks nor log segments found. Creating new log.
I20260925 09:22:09.431886 15270 log.cc:824] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:22:09.431957 15268 log.cc:824] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:22:09.435017 15269 tablet_bootstrap.cc:654] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Neither blocks nor log segments found. Creating new log.
I20260925 09:22:09.436820 15269 log.cc:824] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Log is configured to *not* fsync() on all Append() calls
I20260925 09:22:09.441699 15270 tablet_bootstrap.cc:492] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: No bootstrap required, opened a new log
I20260925 09:22:09.442087 15270 ts_tablet_manager.cc:1397] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.004s
I20260925 09:22:09.451411 15268 tablet_bootstrap.cc:492] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: No bootstrap required, opened a new log
I20260925 09:22:09.451869 15268 ts_tablet_manager.cc:1397] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.013s
I20260925 09:22:09.454317 15270 raft_consensus.cc:348] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.454856 15270 raft_consensus.cc:374] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:22:09.454968 15270 raft_consensus.cc:729] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1d1d190a69423990badbf744e4914e, State: Initialized, Role: FOLLOWER
I20260925 09:22:09.457082 15270 consensus_queue.cc:260] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.459504 15269 tablet_bootstrap.cc:492] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: No bootstrap required, opened a new log
I20260925 09:22:09.460023 15269 ts_tablet_manager.cc:1397] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Time spent bootstrapping tablet: real 0.035s user 0.017s sys 0.010s
I20260925 09:22:09.465708 15268 raft_consensus.cc:348] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.466286 15268 raft_consensus.cc:374] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:22:09.466394 15268 raft_consensus.cc:729] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5ba3a2535b466d9dfa172e3076020f, State: Initialized, Role: FOLLOWER
I20260925 09:22:09.467200 15268 consensus_queue.cc:260] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.470700 15249 heartbeater.cc:503] Master 127.3.82.62:45873 was elected leader, sending a full tablet report...
I20260925 09:22:09.472594 15269 raft_consensus.cc:348] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.473367 15269 raft_consensus.cc:374] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:22:09.473547 15269 raft_consensus.cc:729] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53bcb857b4e42a6940069f24c55dd51, State: Initialized, Role: FOLLOWER
I20260925 09:22:09.474673 15269 consensus_queue.cc:260] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.477341 15270 ts_tablet_manager.cc:1428] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Time spent starting tablet: real 0.035s user 0.013s sys 0.015s
I20260925 09:22:09.483567 15276 raft_consensus.cc:482] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:22:09.483946 15276 raft_consensus.cc:504] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.484215 15115 heartbeater.cc:503] Master 127.3.82.62:45873 was elected leader, sending a full tablet report...
I20260925 09:22:09.484274 15268 ts_tablet_manager.cc:1428] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Time spent starting tablet: real 0.032s user 0.025s sys 0.006s
I20260925 09:22:09.486014 15269 ts_tablet_manager.cc:1428] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Time spent starting tablet: real 0.026s user 0.012s sys 0.004s
I20260925 09:22:09.486126 15276 leader_election.cc:288] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d1d1d190a69423990badbf744e4914e (127.3.82.3:35065), 3c5ba3a2535b466d9dfa172e3076020f (127.3.82.1:43949)
I20260925 09:22:09.495268 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.486926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41547 (local address 127.3.82.1:43949)
0925 09:22:09.487140 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.487147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.488337 (+ 1190us) server_negotiation.cc:407] Connection header received
0925 09:22:09.488602 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.488607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.488672 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.488730 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:22:09.489550 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.490627 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.493268 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.494264 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.494425 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.494544 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.494868 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.495002 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:22:09.495445 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.486820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:43949 (local address 127.3.82.2:41547)
0925 09:22:09.488238 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:09.488258 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:09.488440 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:09.488922 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:09.488926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:09.488949 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:22:09.489363 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.489377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.490803 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.490808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.493076 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.493092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.494483 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.494490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.494669 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.494692 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 09:22:09.494832 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 09:22:09.495010 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"mutex_wait_us":27,"thread_start_us":5263,"threads_started":1}
I20260925 09:22:09.496495 14927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12d476c71544a6aa5fa39c225207ff6" candidate_uuid: "d53bcb857b4e42a6940069f24c55dd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5ba3a2535b466d9dfa172e3076020f" is_pre_election: true
I20260925 09:22:09.497545 14927 raft_consensus.cc:2454] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53bcb857b4e42a6940069f24c55dd51 in term 0.
I20260925 09:22:09.498399 15006 leader_election.cc:302] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c5ba3a2535b466d9dfa172e3076020f, d53bcb857b4e42a6940069f24c55dd51; no voters:
I20260925 09:22:09.498950 15276 raft_consensus.cc:2781] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:22:09.499133 15276 raft_consensus.cc:482] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:22:09.499302 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.488087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:43879 (local address 127.3.82.3:35065)
0925 09:22:09.488311 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.488320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.492514 (+ 4194us) server_negotiation.cc:407] Connection header received
0925 09:22:09.492707 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.492713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.492787 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.492889 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:22:09.493570 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.494633 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.496885 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.498253 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.498407 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.498519 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.498908 (+ 389us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.499047 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:22:09.499300 15276 raft_consensus.cc:3037] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:22:09.499569 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.487292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:35065 (local address 127.3.82.2:43879)
0925 09:22:09.492389 (+ 5097us) negotiation.cc:107] Waiting for socket to connect
0925 09:22:09.492410 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:22:09.492565 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:22:09.492904 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:22:09.492909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:22:09.492932 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:22:09.493387 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.493399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.494772 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.494780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.496654 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:22:09.496667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.498483 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.498490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:22:09.498691 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.498725 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 09:22:09.498880 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 09:22:09.499059 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4980,"mutex_wait_us":3,"thread_start_us":676,"threads_started":1}
I20260925 09:22:09.500690 15204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12d476c71544a6aa5fa39c225207ff6" candidate_uuid: "d53bcb857b4e42a6940069f24c55dd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1d1d190a69423990badbf744e4914e" is_pre_election: true
I20260925 09:22:09.501204 15204 raft_consensus.cc:2454] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d53bcb857b4e42a6940069f24c55dd51 in term 0.
I20260925 09:22:09.502290 15276 raft_consensus.cc:504] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.502936 15276 leader_election.cc:288] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [CANDIDATE]: Term 1 election: Requested vote from peers 8d1d1d190a69423990badbf744e4914e (127.3.82.3:35065), 3c5ba3a2535b466d9dfa172e3076020f (127.3.82.1:43949)
I20260925 09:22:09.503389 15204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12d476c71544a6aa5fa39c225207ff6" candidate_uuid: "d53bcb857b4e42a6940069f24c55dd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1d1d190a69423990badbf744e4914e"
I20260925 09:22:09.503590 15204 raft_consensus.cc:3037] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:22:09.503597 14927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a12d476c71544a6aa5fa39c225207ff6" candidate_uuid: "d53bcb857b4e42a6940069f24c55dd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5ba3a2535b466d9dfa172e3076020f"
I20260925 09:22:09.503796 14927 raft_consensus.cc:3037] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:22:09.506294 15204 raft_consensus.cc:2454] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53bcb857b4e42a6940069f24c55dd51 in term 1.
I20260925 09:22:09.507099 14927 raft_consensus.cc:2454] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d53bcb857b4e42a6940069f24c55dd51 in term 1.
I20260925 09:22:09.507355 15004 leader_election.cc:302] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1d1d190a69423990badbf744e4914e, d53bcb857b4e42a6940069f24c55dd51; no voters:
I20260925 09:22:09.507827 15276 raft_consensus.cc:2781] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:22:09.507982 15276 raft_consensus.cc:686] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 1 LEADER]: Becoming Leader. State: Replica: d53bcb857b4e42a6940069f24c55dd51, State: Running, Role: LEADER
I20260925 09:22:09.508540 15276 consensus_queue.cc:237] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } }
I20260925 09:22:09.512729 14758 catalog_manager.cc:5495] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 reported cstate change: term changed from 0 to 1, leader changed from <none> to d53bcb857b4e42a6940069f24c55dd51 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "d53bcb857b4e42a6940069f24c55dd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53bcb857b4e42a6940069f24c55dd51" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 33837 } health_report { overall_health: HEALTHY } } }
W20260925 09:22:09.526254 15250 tablet.cc:2369] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:22:09.553326 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.546583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37322 (local address 127.3.82.2:33837)
0925 09:22:09.546788 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.546795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.546934 (+ 139us) server_negotiation.cc:407] Connection header received
0925 09:22:09.547186 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.547192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.547334 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.547514 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:22:09.547994 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.549146 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.550650 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.551266 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.551390 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.551939 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:22:09.552652 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:22:09.552812 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.552932 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.553019 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 09:22:09.562678 15204 raft_consensus.cc:1261] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 1 FOLLOWER]: Refusing update from remote peer d53bcb857b4e42a6940069f24c55dd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:22:09.562808 14927 raft_consensus.cc:1261] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 1 FOLLOWER]: Refusing update from remote peer d53bcb857b4e42a6940069f24c55dd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:22:09.563782 15276 consensus_queue.cc:1035] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1d1d190a69423990badbf744e4914e" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 35065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:22:09.564142 15277 consensus_queue.cc:1035] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c5ba3a2535b466d9dfa172e3076020f" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:22:09.583779 15282 mvcc.cc:205] Tried to move back new op lower bound from 7333184018678280192 to 7333184018471550976. Current Snapshot: MvccSnapshot[applied={T|T < 7333184018678280192}]
I20260925 09:22:09.598317 15284 mvcc.cc:205] Tried to move back new op lower bound from 7333184018678280192 to 7333184018471550976. Current Snapshot: MvccSnapshot[applied={T|T < 7333184018678280192 or (T in {7333184018678280192})}]
I20260925 09:22:09.603084 14759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:22:09.603392 14759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:22:09.607671 14759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c9a9df26c6943fca44f3d979dce4be0: Sending DeleteTablet for 3 replicas of tablet a12d476c71544a6aa5fa39c225207ff6
I20260925 09:22:09.608942 15184 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d476c71544a6aa5fa39c225207ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:22:09 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:54518
I20260925 09:22:09.609071 14907 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d476c71544a6aa5fa39c225207ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:22:09 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:54814
I20260925 09:22:09.610181 15050 tablet_service.cc:1514] Processing DeleteTablet for tablet a12d476c71544a6aa5fa39c225207ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:22:09 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:37310
I20260925 09:22:09.611807 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 14839
I20260925 09:22:09.637743 15293 tablet_replica.cc:330] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: stopping tablet replica
I20260925 09:22:09.638345 15293 raft_consensus.cc:2229] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:22:09.639051 15293 raft_consensus.cc:2258] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:22:09.639915 15292 tablet_replica.cc:330] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: stopping tablet replica
I20260925 09:22:09.640386 15292 raft_consensus.cc:2229] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:22:09.640900 15292 raft_consensus.cc:2258] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:22:09.646440 15291 tablet_replica.cc:330] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: stopping tablet replica
I20260925 09:22:09.646956 15291 raft_consensus.cc:2229] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:22:09.647540 15291 raft_consensus.cc:2258] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:22:09.649931 15292 ts_tablet_manager.cc:1905] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:22:09.649955 15291 ts_tablet_manager.cc:1905] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:22:09.651216 15293 ts_tablet_manager.cc:1905] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:22:09.658174 15291 ts_tablet_manager.cc:1918] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:22:09.658490 15291 log.cc:1196] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/a12d476c71544a6aa5fa39c225207ff6
I20260925 09:22:09.659135 15291 ts_tablet_manager.cc:1939] T a12d476c71544a6aa5fa39c225207ff6 P 8d1d1d190a69423990badbf744e4914e: Deleting consensus metadata
I20260925 09:22:09.661350 14744 catalog_manager.cc:4841] TS 8d1d1d190a69423990badbf744e4914e (127.3.82.3:35065): tablet a12d476c71544a6aa5fa39c225207ff6 (table test-table [id=34006516f6044f6abb9d8b568a221e58]) successfully deleted
Sep 25 09:22:09 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:22:09.667162 15292 ts_tablet_manager.cc:1918] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:22:09.667413 15292 log.cc:1196] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/a12d476c71544a6aa5fa39c225207ff6
I20260925 09:22:09.668046 15292 ts_tablet_manager.cc:1939] T a12d476c71544a6aa5fa39c225207ff6 P 3c5ba3a2535b466d9dfa172e3076020f: Deleting consensus metadata
I20260925 09:22:09.670264 14746 catalog_manager.cc:4841] TS 3c5ba3a2535b466d9dfa172e3076020f (127.3.82.1:43949): tablet a12d476c71544a6aa5fa39c225207ff6 (table test-table [id=34006516f6044f6abb9d8b568a221e58]) successfully deleted
I20260925 09:22:09.672302 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.613711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.3.82.1:43949)
0925 09:22:09.613923 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.613930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.633012 (+ 19082us) server_negotiation.cc:407] Connection header received
0925 09:22:09.634072 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.634078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.634195 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.634285 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:09.641406 (+ 7121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.642539 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.656420 (+ 13881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.657133 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.657257 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.663014 (+ 5757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:09.663032 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:09.663051 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:09.663124 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:09.668230 (+ 5106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.669268 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.669279 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.669283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.669479 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.669996 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.670002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.670005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.670483 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:09.670758 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.671427 (+ 669us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.671706 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:22:09.673346 14967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:22:09.676908 15293 ts_tablet_manager.cc:1918] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:22:09.677135 15293 log.cc:1196] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/a12d476c71544a6aa5fa39c225207ff6
I20260925 09:22:09.681073 15293 ts_tablet_manager.cc:1939] T a12d476c71544a6aa5fa39c225207ff6 P d53bcb857b4e42a6940069f24c55dd51: Deleting consensus metadata
I20260925 09:22:09.683482 14743 catalog_manager.cc:4841] TS d53bcb857b4e42a6940069f24c55dd51 (127.3.82.2:33837): tablet a12d476c71544a6aa5fa39c225207ff6 (table test-table [id=34006516f6044f6abb9d8b568a221e58]) successfully deleted
I20260925 09:22:09.806944 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 14839
I20260925 09:22:09.825587 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 14978
Sep 25 09:22:09 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:22:09.844715 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.826647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.3.82.2:33837)
0925 09:22:09.826885 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.826896 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.826920 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:22:09.827022 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.827028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.827114 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.827239 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:09.828502 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.829976 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.831418 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.832172 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.832313 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.837240 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:09.837261 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:09.837353 (+ 92us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:09.837411 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:09.841093 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.842187 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.842196 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.842198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.842405 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.842859 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.842866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.842868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.843245 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:09.843587 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:09.843999 (+ 412us) server_negotiation.cc:299] Negotiation successful
0925 09:22:09.844244 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:22:09.845621 15110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:22:09.964679 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 14978
I20260925 09:22:09.983438 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 15121
Sep 25 09:22:09 dist-test-slave-4wm5 krb5kdc[14283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:22:10.001675 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:09.984341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.3.82.3:35065)
0925 09:22:09.984506 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:09.984511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:09.984535 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:22:09.984832 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:09.984836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:09.984904 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:09.984995 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:09.986561 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.987573 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.989445 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:09.990030 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:09.990184 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:09.995076 (+ 4892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:09.995095 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:09.995112 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:09.995161 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:09.998522 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:09.999428 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:09.999436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:09.999439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:09.999588 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:10.000057 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:10.000061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:10.000064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:10.000451 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:10.000846 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:10.001173 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 09:22:10.001370 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 09:22:10.002496 15244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:22:10.096505 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 15121
I20260925 09:22:10.114859 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 14727
I20260925 09:22:10.116039 14789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:22:10.232398 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 14727
I20260925 09:22:10.258299 3400 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 09:22:10.605127 3400 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 09:22:10.605526 3400 mini_ranger.cc:66] Stopping Ranger...
I20260925 09:22:10.952255 3400 mini_ranger.cc:68] Stopped Ranger
2026-09-25 09:22:10.952 UTC [14313] LOG: received smart shutdown request
2026-09-25 09:22:10.956 UTC [14313] LOG: background worker "logical replication launcher" (PID 14322) exited with exit code 1
2026-09-25 09:22:10.957 UTC [14317] LOG: shutting down
2026-09-25 09:22:11.191 UTC [14313] LOG: database system is shut down
2026-09-25T09:22:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53163 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:22:11 dist-test-slave-4wm5 krb5kdc[15307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:22:11 dist-test-slave-4wm5 krb5kdc[15307](info): set up 2 sockets
Sep 25 09:22:11 dist-test-slave-4wm5 krb5kdc[15307](info): commencing operation
krb5kdc: starting...
W20260925 09:22:12.538640 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:22:12 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:22:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:22:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/rangeradmin_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/rangeradmin_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/rangerlookup_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/rangerlookup_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
I20260925 09:22:12.906458 3400 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0mddRs/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/postgres -l logfile start
2026-09-25 09:22:15.795 UTC [15337] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 09:22:15.796 UTC [15337] LOG: listening on IPv4 address "127.3.82.20", port 44603
2026-09-25 09:22:15.800 UTC [15337] LOG: listening on Unix socket "/tmp/.s.PGSQL.44603"
2026-09-25 09:22:15.825 UTC [15340] LOG: database system was shut down at 2026-09-25 09:22:14 UTC
2026-09-25 09:22:15.830 UTC [15337] LOG: database system is ready to accept connections
I20260925 09:22:16.806237 3400 mini_postgres.cc:96] Postgres bound to 44603
2026-09-25 09:22:16.825 UTC [15348] FATAL: database "slave" does not exist
127.3.82.20:44603 - accepting connections
I20260925 09:22:16.826090 3400 mini_ranger.cc:161] Starting Ranger...
I20260925 09:22:16.854789 3400 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 09:22:17.459178 3400 mini_ranger.cc:87] Created ranger Postgres database
I20260925 09:22:17.459300 3400 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-admin
2026-09-25 09:22:17,755 [I] DB FLAVOR :POSTGRES
2026-09-25 09:22:17,756 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:22:17,756 [I] Checking connection..
2026-09-25 09:22:17,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 09:22:18,024 [I] Checking connection passed.
2026-09-25 09:22:18,024 [I] --------- Verifying version history table ---------
2026-09-25 09:22:18,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:22:18,276 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:22:18,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:22:18,547 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 09:22:18,548 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 09:22:18,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 09:22:18.818 UTC [15412] WARNING: there is no transaction in progress
2026-09-25 09:22:18,825 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 09:22:18,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 09:22:19,067 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 09:22:19,067 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:22:19,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 09:22:19,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 09:22:19,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4wm5.c.gcp-upstream.internal','N') ;"
2026-09-25 09:22:19,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:22:20,131 [I] Table x_portal_user does not exist in database ranger
2026-09-25 09:22:20,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:22:20,400 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 09:22:20,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:22:20,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:22:20,925 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 09:22:20,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 09:22:22.077 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.097 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.117 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.136 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.156 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.177 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.240 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.248 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.266 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.278 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.292 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.308 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.319 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.327 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.335 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:22.344 UTC [15520] WARNING: there is no transaction in progress
2026-09-25 09:22:23,148 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 09:22:23,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 09:22:23,434 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 09:22:23,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 09:22:23,705 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 09:22:23,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 09:22:23,977 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 09:22:23,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 09:22:24,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 09:22:24,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4wm5.c.gcp-upstream.internal';"
2026-09-25 09:22:24,775 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 09:22:24,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 09:22:25,059 [I] DB_PATCHES have already been applied
I20260925 09:22:25.066537 3400 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-admin:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0mddRs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 09:22:25.066636 3400 mini_ranger.cc:193] Using host: 127.3.82.20
I20260925 09:22:25.082525 3400 mini_ranger.cc:227] Ranger admin URL: http://127.3.82.20:33885
Sep 25, 2026 9:22:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:22:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:22:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 9:22:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 9:22:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 9:22:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 9:22:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 09:22:26.280556 3400 mini_ranger.cc:160] Time spent starting Ranger: real 9.454s user 0.005s sys 0.039s
Sep 25, 2026 9:22:47 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 9:22:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 9:22:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 9:22:47 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 9:22:48 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 09:22:49.067296 3400 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.82.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.82.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/rangerkms_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.82.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/rangerkms_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.82.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.82.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.82.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/HTTP_127.3.82.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 09:22:49 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328169, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 09:22:49.259244 3400 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 09:22:49.284261 3400 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 09:22:50.370942 3400 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 09:22:50.371080 3400 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms
I20260925 09:22:50.371130 3400 mini_ranger_kms.cc:227] Using postgres at 127.3.82.20:44603
2026-09-25 09:22:50,736 [I] DB FLAVOR :POSTGRES
2026-09-25 09:22:50,736 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 09:22:50,736 [I] Checking connection
2026-09-25 09:22:50,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 09:22:51,010 [I] connection success
2026-09-25 09:22:51,010 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 09:22:51,010 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 09:22:51,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 09:22:51,290 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 09:22:51,291 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 09:22:51,291 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 09:22:51,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.82.20:44603/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 09:22:51,572 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 09:22:52.152076 3400 mini_ranger_kms.cc:310] Created kms service
I20260925 09:22:52.355093 3400 mini_ranger_kms.cc:326] Created kudu user
I20260925 09:22:52.407995 3400 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 09:22:52 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328169, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:52.828198 3400 mini_ranger_kms.cc:384] Added ranger policy
I20260925 09:22:52.828568 3400 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms:/tmp/dist-test-task0mddRs/build/asan/bin/postgresql.jar:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0mddRs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 09:22:52.828636 3400 mini_ranger_kms.cc:240] Using host: 127.3.82.20
I20260925 09:22:52.848093 3400 mini_ranger_kms.cc:276] Ranger KMS PID: 15701
I20260925 09:22:52.848246 3400 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.82.20:55057
Sep 25, 2026 9:22:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 9:22:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 9:22:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 9:22:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 9:22:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 9:22:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 09:22:55 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.82.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:22:55.232676 3400 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.973s user 0.076s sys 0.308s
I20260925 09:22:55.232918 3400 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 09:22:55.233098 3400 mini_ranger_kms.cc:401] 127.3.82.20:55057/kms/v1/keys
Sep 25 09:22:55 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.82.20@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
Sep 25 09:22:56 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.82.20@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
Sep 25 09:22:58 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:22:58.591297 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42259
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42259
--ranger_config_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:22:58.952306 15751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:22:58.952616 15751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:22:58.952762 15751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:22:58.960436 15751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:22:58.960574 15751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:22:58.960620 15751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:22:58.960728 15751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:22:58.960812 15751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:22:58.971805 15751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.82.20:55057/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42259
--ranger_config_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:22:58.973248 15751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:22:58.974920 15751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:22:58.988185 15759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:58.988206 15757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:22:58.989010 15756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:22:58.989496 15751 server_base.cc:1034] running on GCE node
I20260925 09:22:58.990463 15751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:22:58.992223 15751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:22:58.993595 15751 hybrid_clock.cc:648] HybridClock initialized: now 1790328178993531 us; error 48 us; skew 500 ppm
I20260925 09:22:58.994079 15751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:22:58 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:22:58.997735 15751 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:22:58.999701 15751 webserver.cc:460] Webserver started at http://127.3.82.62:41373/ using document root <none> and password file <none>
I20260925 09:22:59.000474 15751 fs_manager.cc:359] Metadata directory not provided
I20260925 09:22:59.000602 15751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:22:59.000972 15751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:22:59 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:22:59.065263 15751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "51b955921daa4e9fa596b31f257d7a76"
format_stamp: "Formatted at 2026-09-25 09:22:59 on dist-test-slave-4wm5"
server_key: "25f6f8758e0fc801d29ca36ca84707b4"
server_key_iv: "463c117c1fa3d09b9513757a16a4716d"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:59.066749 15751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "51b955921daa4e9fa596b31f257d7a76"
format_stamp: "Formatted at 2026-09-25 09:22:59 on dist-test-slave-4wm5"
server_key: "25f6f8758e0fc801d29ca36ca84707b4"
server_key_iv: "463c117c1fa3d09b9513757a16a4716d"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:59.074715 15751 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260925 09:22:59.096810 15766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:22:59.098976 15751 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:22:59.099216 15751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "51b955921daa4e9fa596b31f257d7a76"
format_stamp: "Formatted at 2026-09-25 09:22:59 on dist-test-slave-4wm5"
server_key: "25f6f8758e0fc801d29ca36ca84707b4"
server_key_iv: "463c117c1fa3d09b9513757a16a4716d"
server_key_version: "kuduclusterkey@0"
I20260925 09:22:59.099501 15751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:22:59.138151 15751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:22:59.143415 15751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:22:59.143822 15751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:22:59.169576 15751 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:42259
I20260925 09:22:59.169600 15817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:42259 every 8 connection(s)
I20260925 09:22:59.171176 15751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:22:59.176422 15818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:22:59.178656 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 15751
I20260925 09:22:59.178887 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:22:59.179603 3400 external_mini_cluster.cc:1428] Setting key 0fdcd25fa425e22bf8b68946826d2d9e
I20260925 09:22:59.192626 15818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Bootstrap starting.
I20260925 09:22:59.197755 15818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Neither blocks nor log segments found. Creating new log.
I20260925 09:22:59.199128 15818 log.cc:824] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Log is configured to *not* fsync() on all Append() calls
I20260925 09:22:59.202505 15818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: No bootstrap required, opened a new log
Sep 25 09:22:59 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:22:59.213560 15818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:22:59.214097 15818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:22:59.214231 15818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b955921daa4e9fa596b31f257d7a76, State: Initialized, Role: FOLLOWER
I20260925 09:22:59.214972 15818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:22:59.215237 15818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:22:59.215384 15818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:22:59.215626 15818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:22:59.218400 15818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:22:59.218632 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:22:59.182070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.3.82.62:42259)
0925 09:22:59.193338 (+ 11268us) server_negotiation.cc:206] Beginning negotiation
0925 09:22:59.193356 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:22:59.193948 (+ 592us) server_negotiation.cc:407] Connection header received
0925 09:22:59.195978 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:22:59.196002 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:22:59.196432 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:22:59.196856 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:22:59.198449 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:59.200027 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:59.202056 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:22:59.203110 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:22:59.203322 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:22:59.208539 (+ 5217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:22:59.208551 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:22:59.208575 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:22:59.208632 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:22:59.212871 (+ 4239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:59.213809 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:59.213818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:59.213868 (+ 50us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:59.214064 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:22:59.214599 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:22:59.214605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:22:59.214609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:22:59.215375 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:22:59.215715 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0925 09:22:59.216681 (+ 966us) server_negotiation.cc:299] Negotiation successful
0925 09:22:59.217171 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10981,"thread_start_us":10709,"threads_started":1}
I20260925 09:22:59.219074 15818 leader_election.cc:302] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51b955921daa4e9fa596b31f257d7a76; no voters:
I20260925 09:22:59.219797 15818 leader_election.cc:288] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:22:59.219914 15823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:22:59.222285 15818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:22:59.222987 15823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 LEADER]: Becoming Leader. State: Replica: 51b955921daa4e9fa596b31f257d7a76, State: Running, Role: LEADER
I20260925 09:22:59.223745 15823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:22:59.227231 15825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51b955921daa4e9fa596b31f257d7a76. Latest consensus state: current_term: 1 leader_uuid: "51b955921daa4e9fa596b31f257d7a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } } }
I20260925 09:22:59.227761 15825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: This master's current role is: LEADER
I20260925 09:22:59.228418 15824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51b955921daa4e9fa596b31f257d7a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } } }
I20260925 09:22:59.228731 15824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: This master's current role is: LEADER
I20260925 09:22:59.228734 15831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:22:59.241205 15831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:22:59.259409 15818 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 09:22:59.260720 15831 catalog_manager.cc:1347] Generated new cluster ID: 9e92e16253e14b1ba9f17014097425e5
I20260925 09:22:59.260840 15831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:22:59.289290 15831 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:22:59.291693 15831 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:22:59.301378 15831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Generated new TSK 0
I20260925 09:22:59.302704 15831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:23:00.183239 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:42259
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:23:00 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 09:23:00.622627 15864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:00.623052 15864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:00.623267 15864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:00.632540 15864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:00.632670 15864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:00.632853 15864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:23:00.645373 15864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.82.20:55057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:42259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:00.647126 15864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:00.649133 15864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:00.670302 15874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:00.670964 15875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:00.679894 15864 server_base.cc:1034] running on GCE node
W20260925 09:23:00.681178 15878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:00.682215 15864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:00.683717 15864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:00.685070 15864 hybrid_clock.cc:648] HybridClock initialized: now 1790328180684996 us; error 66 us; skew 500 ppm
I20260925 09:23:00.685505 15864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:00 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:00.690429 15864 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:23:00.692870 15864 webserver.cc:460] Webserver started at http://127.3.82.1:32961/ using document root <none> and password file <none>
I20260925 09:23:00.694080 15864 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:00.694244 15864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:00.694672 15864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:23:00 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:00.722661 15864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "75e727d255ed4379bcdcbb12329cf88e"
format_stamp: "Formatted at 2026-09-25 09:23:00 on dist-test-slave-4wm5"
server_key: "acd6e4e343eab74bd636898d9033d0ed"
server_key_iv: "4aca4015e1e22970e10a64c15e2b8ace"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:00.723991 15864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "75e727d255ed4379bcdcbb12329cf88e"
format_stamp: "Formatted at 2026-09-25 09:23:00 on dist-test-slave-4wm5"
server_key: "acd6e4e343eab74bd636898d9033d0ed"
server_key_iv: "4aca4015e1e22970e10a64c15e2b8ace"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:00.732304 15864 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.006s
I20260925 09:23:00.766448 15884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:00.768433 15864 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.003s
I20260925 09:23:00.768708 15864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "75e727d255ed4379bcdcbb12329cf88e"
format_stamp: "Formatted at 2026-09-25 09:23:00 on dist-test-slave-4wm5"
server_key: "acd6e4e343eab74bd636898d9033d0ed"
server_key_iv: "4aca4015e1e22970e10a64c15e2b8ace"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:00.769019 15864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:00.793782 15864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:00.798622 15864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:00.798974 15864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:00.800143 15864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:00.802871 15864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:00.803004 15864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:00.803138 15864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:00.803202 15864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:00.932351 15864 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:39887
I20260925 09:23:00.932390 15996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:39887 every 8 connection(s)
I20260925 09:23:00.934580 15864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:23:00.943665 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 15864
I20260925 09:23:00.943909 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:23:00.944727 3400 external_mini_cluster.cc:1428] Setting key 86fccec969c09d61fc1ca3a7ba19fac7
Sep 25 09:23:00 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:23:00.987546 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:00.938321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:36173 (local address 127.3.82.62:42259)
0925 09:23:00.941957 (+ 3636us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:00.941964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:00.957897 (+ 15933us) server_negotiation.cc:407] Connection header received
0925 09:23:00.962290 (+ 4393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:00.962298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:00.962443 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:00.962546 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:00.965348 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:00.966658 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:00.969694 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:00.970435 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:00.970562 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:00.978725 (+ 8163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:00.978733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:00.978738 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:00.978793 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:00.982909 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:00.984089 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:00.984097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:00.984101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:00.984272 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:00.985014 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:00.985020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:00.985024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:00.985442 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:00.985576 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:00.986868 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0925 09:23:00.987124 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3476,"thread_start_us":3339,"threads_started":1}
I20260925 09:23:00.990477 16000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:00.939486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42259 (local address 127.3.82.1:36173)
0925 09:23:00.957590 (+ 18104us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:00.957668 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:00.961781 (+ 4113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:00.963244 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:00.963257 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:00.964006 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:00.964995 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:00.965034 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:00.966910 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:00.966916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:00.969449 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:00.969480 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:00.970982 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:00.970992 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:00.971326 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:00.972260 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:00.972294 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:00.978330 (+ 6036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:00.983175 (+ 4845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:00.983188 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:00.983210 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:00.983914 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:00.984444 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:00.984449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:00.984452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:00.984865 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:00.985697 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:00.985706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:00.986299 (+ 593us) client_negotiation.cc:769] Sending connection context
0925 09:23:00.988171 (+ 1872us) client_negotiation.cc:240] Negotiation successful
0925 09:23:00.988818 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17685,"thread_start_us":17374,"threads_started":1}
I20260925 09:23:00.993134 15997 heartbeater.cc:347] Connected to a master server at 127.3.82.62:42259
I20260925 09:23:00.993525 15997 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:00.994534 15997 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
I20260925 09:23:00.997995 15783 ts_manager.cc:194] Registered new tserver with Master: 75e727d255ed4379bcdcbb12329cf88e (127.3.82.1:39887)
I20260925 09:23:01.000257 15783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:36173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:23:01.027084 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:42259
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:23:01 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
Sep 25 09:23:01 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
W20260925 09:23:01.560974 16003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:01.561525 16003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:01.561743 16003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:01.573560 16003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:01.573855 16003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:01.574115 16003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:23:01.590451 16003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.82.20:55057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:42259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:01.592530 16003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:01.594456 16003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:01.613732 16010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:01.614355 16011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:01.622709 16003 server_base.cc:1034] running on GCE node
W20260925 09:23:01.623929 16014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:01.625022 16003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:01.629760 16003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:01.631151 16003 hybrid_clock.cc:648] HybridClock initialized: now 1790328181631070 us; error 83 us; skew 500 ppm
I20260925 09:23:01.631714 16003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:01 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328181, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:01.636763 16003 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:23:01.650424 16003 webserver.cc:460] Webserver started at http://127.3.82.2:38379/ using document root <none> and password file <none>
I20260925 09:23:01.651485 16003 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:01.651726 16003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:01.652168 16003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:23:01 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328181, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:01.671751 16003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "955274ee3d694643b6ce9743b9cbe09d"
format_stamp: "Formatted at 2026-09-25 09:23:01 on dist-test-slave-4wm5"
server_key: "6323e147c9e34100d093e2fcafe72f8b"
server_key_iv: "b6921df7f916341074f72f9f8e0e574e"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:01.673041 16003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "955274ee3d694643b6ce9743b9cbe09d"
format_stamp: "Formatted at 2026-09-25 09:23:01 on dist-test-slave-4wm5"
server_key: "6323e147c9e34100d093e2fcafe72f8b"
server_key_iv: "b6921df7f916341074f72f9f8e0e574e"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:01.687654 16003 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.003s sys 0.008s
I20260925 09:23:01.714604 16020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:01.716933 16003 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.005s
I20260925 09:23:01.717137 16003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "955274ee3d694643b6ce9743b9cbe09d"
format_stamp: "Formatted at 2026-09-25 09:23:01 on dist-test-slave-4wm5"
server_key: "6323e147c9e34100d093e2fcafe72f8b"
server_key_iv: "b6921df7f916341074f72f9f8e0e574e"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:01.717408 16003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:01.767271 16003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:01.771895 16003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:01.772286 16003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:01.773455 16003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:01.775158 16003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:01.775267 16003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:01.775416 16003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:01.775488 16003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:01.811764 16003 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:45259
I20260925 09:23:01.811806 16139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:45259 every 8 connection(s)
I20260925 09:23:01.813151 16003 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:23:01.819072 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16003
I20260925 09:23:01.819260 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:23:01.819792 3400 external_mini_cluster.cc:1428] Setting key 4909cb6de3c96b2afab9c8d685cd05a1
Sep 25 09:23:01 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328181, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:01.840077 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:01.816041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41209 (local address 127.3.82.62:42259)
0925 09:23:01.816549 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:01.816553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:01.817560 (+ 1007us) server_negotiation.cc:407] Connection header received
0925 09:23:01.820071 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:01.820075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:01.820158 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:01.820252 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:01.822234 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:01.823269 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:01.826258 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:01.826733 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:01.826850 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:01.831706 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:01.831713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:01.831716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:01.831764 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:01.835442 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:01.836708 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:01.836715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:01.836718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:01.836913 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:01.837528 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:01.837535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:01.837537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:01.837990 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:01.838209 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:01.839222 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0925 09:23:01.839547 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":287,"threads_started":1}
I20260925 09:23:01.841305 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:01.816580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42259 (local address 127.3.82.2:41209)
0925 09:23:01.817357 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:01.817404 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:01.819638 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:01.820582 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:01.820592 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:01.821230 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:01.822042 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:01.822066 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:01.823490 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:01.823499 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:01.826007 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:01.826061 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:01.827009 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:01.827020 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:01.827390 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:01.828290 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:01.828325 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:01.831382 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:01.835709 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:01.835722 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:01.835740 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:01.836503 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:01.837071 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:01.837078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:01.837083 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:01.837354 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:01.838158 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:01.838169 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:01.838749 (+ 580us) client_negotiation.cc:769] Sending connection context
0925 09:23:01.839091 (+ 342us) client_negotiation.cc:240] Negotiation successful
0925 09:23:01.839764 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":330,"threads_started":1}
I20260925 09:23:01.843403 16140 heartbeater.cc:347] Connected to a master server at 127.3.82.62:42259
I20260925 09:23:01.843798 16140 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:01.844664 16140 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:23:01.846657 15783 ts_manager.cc:194] Registered new tserver with Master: 955274ee3d694643b6ce9743b9cbe09d (127.3.82.2:45259)
I20260925 09:23:01.847527 15783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:41209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:23:01.881930 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:42259
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:02.005162 15997 heartbeater.cc:503] Master 127.3.82.62:42259 was elected leader, sending a full tablet report...
W20260925 09:23:02.243413 16146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:02.244010 16146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:02.244197 16146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:02.252875 16146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:02.253087 16146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:02.253278 16146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:23:02.263741 16146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.82.20:55057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:42259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:02.265200 16146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:02.266768 16146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:02.281733 16152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:02.281677 16154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:02.281677 16151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:02.283380 16146 server_base.cc:1034] running on GCE node
I20260925 09:23:02.284044 16146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:02.285053 16146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:02.286362 16146 hybrid_clock.cc:648] HybridClock initialized: now 1790328182286286 us; error 68 us; skew 500 ppm
I20260925 09:23:02.286759 16146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:02 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328182, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:02.290302 16146 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:23:02.291980 16146 webserver.cc:460] Webserver started at http://127.3.82.3:34563/ using document root <none> and password file <none>
I20260925 09:23:02.292762 16146 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:02.292919 16146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:02.293286 16146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 09:23:02 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328182, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:02.309243 16146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "898f465898544e9496ce3ff02eb96b82"
format_stamp: "Formatted at 2026-09-25 09:23:02 on dist-test-slave-4wm5"
server_key: "5489d4a9255c4a8813aa29f63612f962"
server_key_iv: "5135ac8fcc19bdcd8c795c6ec2dd5fb5"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:02.310231 16146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "898f465898544e9496ce3ff02eb96b82"
format_stamp: "Formatted at 2026-09-25 09:23:02 on dist-test-slave-4wm5"
server_key: "5489d4a9255c4a8813aa29f63612f962"
server_key_iv: "5135ac8fcc19bdcd8c795c6ec2dd5fb5"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:02.316079 16146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:23:02.333453 16161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:02.335532 16146 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:23:02.335867 16146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "898f465898544e9496ce3ff02eb96b82"
format_stamp: "Formatted at 2026-09-25 09:23:02 on dist-test-slave-4wm5"
server_key: "5489d4a9255c4a8813aa29f63612f962"
server_key_iv: "5135ac8fcc19bdcd8c795c6ec2dd5fb5"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:02.336126 16146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:02.359817 16146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:02.366209 16146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:02.366607 16146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:02.368037 16146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:02.369971 16146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:02.370052 16146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:02.370204 16146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:02.370280 16146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:02.409296 16146 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:41063
I20260925 09:23:02.409351 16273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:41063 every 8 connection(s)
I20260925 09:23:02.410812 16146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:23:02.419283 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16146
I20260925 09:23:02.419502 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:23:02.420008 3400 external_mini_cluster.cc:1428] Setting key 7ea3fe830f7660a2398003dc1c38d348
Sep 25 09:23:02 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328182, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:02.434139 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:02.413702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:46503 (local address 127.3.82.62:42259)
0925 09:23:02.414308 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:02.414313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:02.415339 (+ 1026us) server_negotiation.cc:407] Connection header received
0925 09:23:02.417352 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:02.417358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:02.417477 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:02.417611 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:02.419460 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.420296 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.422376 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.422960 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.423101 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:02.427157 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:02.427163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:02.427167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:02.427207 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:02.430316 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.431254 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.431260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.431262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.431408 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.432019 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.432024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.432027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.432341 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:02.432514 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:02.433449 (+ 935us) server_negotiation.cc:299] Negotiation successful
0925 09:23:02.433679 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":351,"threads_started":1}
I20260925 09:23:02.436088 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:02.414361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42259 (local address 127.3.82.3:46503)
0925 09:23:02.415148 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:02.415192 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:02.417036 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:02.417956 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:02.417967 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:02.418435 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:02.419227 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:02.419253 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.420457 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.420463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:02.422217 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:02.422231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.423207 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.423215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:02.423530 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:02.424358 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:02.424385 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:02.426783 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:02.430542 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:02.430550 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:02.430567 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:02.431090 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:02.431551 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:02.431555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:02.431557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:02.431794 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:02.432499 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:02.432507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:02.432938 (+ 431us) client_negotiation.cc:769] Sending connection context
0925 09:23:02.433320 (+ 382us) client_negotiation.cc:240] Negotiation successful
0925 09:23:02.434317 (+ 997us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":326,"threads_started":1}
I20260925 09:23:02.437563 16274 heartbeater.cc:347] Connected to a master server at 127.3.82.62:42259
I20260925 09:23:02.437943 16274 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:02.438757 16274 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
I20260925 09:23:02.440338 15783 ts_manager.cc:194] Registered new tserver with Master: 898f465898544e9496ce3ff02eb96b82 (127.3.82.3:41063)
I20260925 09:23:02.441310 15783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:46503
I20260925 09:23:02.449884 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:23:02.451707 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 15864
Sep 25 09:23:02 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:23:02.472411 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:02.452861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.3.82.1:39887)
0925 09:23:02.453792 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:02.453800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:02.453885 (+ 85us) server_negotiation.cc:407] Connection header received
0925 09:23:02.453966 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:02.453972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:02.454206 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:02.454359 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:02.456183 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.457411 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.459370 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.460080 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.460203 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:02.465087 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:02.465107 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:02.465131 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:02.465190 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:02.469025 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.470042 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.470054 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.470057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.470272 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.470714 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.470718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.470720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.471163 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:02.471412 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:02.471759 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 09:23:02.472106 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":445,"threads_started":1}
I20260925 09:23:02.473973 15992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:02.561622 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 15864
I20260925 09:23:02.577929 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16003
Sep 25 09:23:02 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:23:02.595908 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:02.578902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.3.82.2:45259)
0925 09:23:02.579499 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:02.579505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:02.579533 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:23:02.579593 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:02.579599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:02.579810 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:02.579951 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:02.581213 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.582479 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.583971 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.584556 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.584678 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:02.589743 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:02.589765 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:02.589790 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:02.589866 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:02.592953 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.593750 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.593757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.593759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.593959 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.594325 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.594328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.594330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.594676 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:02.594892 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:02.595235 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 09:23:02.595440 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":316,"threads_started":1}
I20260925 09:23:02.597661 16135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:02.684546 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16003
I20260925 09:23:02.701586 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16146
Sep 25 09:23:02 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:23:02.719795 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:02.702861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.3.82.3:41063)
0925 09:23:02.703564 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:02.703584 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:02.703613 (+ 29us) server_negotiation.cc:407] Connection header received
0925 09:23:02.703681 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:02.703687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:02.703893 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:02.704033 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:02.705460 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.706688 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.707994 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:02.708583 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:02.708679 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:02.713506 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:02.713528 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:02.713544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:02.713587 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:02.716419 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.717264 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.717272 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.717274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.717477 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:02.718231 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:02.718235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:02.718238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:02.718659 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:02.718915 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:02.719292 (+ 377us) server_negotiation.cc:299] Negotiation successful
0925 09:23:02.719499 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":409,"threads_started":1}
I20260925 09:23:02.721256 16269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:02.806486 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16146
I20260925 09:23:02.822492 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 15751
I20260925 09:23:02.823900 15813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:02.934043 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 15751
I20260925 09:23:02.957213 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42259
--webserver_interface=127.3.82.62
--webserver_port=41373
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42259
--ranger_config_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:03.319602 16286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:03.319928 16286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:03.320109 16286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:03.327908 16286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:03.328068 16286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:03.328200 16286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:03.328307 16286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:03.328392 16286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:03.339150 16286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42259
--ranger_config_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=41373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:03.340536 16286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:03.342072 16286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:03.355259 16291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:03.355309 16292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:03.355283 16294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:03.356570 16286 server_base.cc:1034] running on GCE node
I20260925 09:23:03.357400 16286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:03.358956 16286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:03.360170 16286 hybrid_clock.cc:648] HybridClock initialized: now 1790328183360106 us; error 76 us; skew 500 ppm
I20260925 09:23:03.360513 16286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:03 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328183, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:03.364140 16286 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:03.365950 16286 webserver.cc:460] Webserver started at http://127.3.82.62:41373/ using document root <none> and password file <none>
I20260925 09:23:03.366679 16286 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:03.366796 16286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:23:03 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328183, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:03.430121 16286 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:23:03.433984 16302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:03.436061 16286 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:23:03.436316 16286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "51b955921daa4e9fa596b31f257d7a76"
format_stamp: "Formatted at 2026-09-25 09:22:59 on dist-test-slave-4wm5"
server_key: "25f6f8758e0fc801d29ca36ca84707b4"
server_key_iv: "463c117c1fa3d09b9513757a16a4716d"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:03.437451 16286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:03.452175 16286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:03.457154 16286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:03.457439 16286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:03.477810 16286 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:42259
I20260925 09:23:03.477878 16353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:42259 every 8 connection(s)
I20260925 09:23:03.479518 16286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:03.485719 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16286
I20260925 09:23:03.486923 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:39887
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:42259
--webserver_port=32961
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:03.488034 16354 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:23:03.493876 16354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Bootstrap starting.
I20260925 09:23:03.515034 16354 log.cc:824] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:03.525398 16354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:23:03.526062 16354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Bootstrap complete.
I20260925 09:23:03.535672 16354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:23:03.536597 16354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b955921daa4e9fa596b31f257d7a76, State: Initialized, Role: FOLLOWER
I20260925 09:23:03.537366 16354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:23:03.537608 16354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:23:03.537729 16354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:23:03.537956 16354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:23:03.542273 16354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:23:03.542984 16354 leader_election.cc:302] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51b955921daa4e9fa596b31f257d7a76; no voters:
I20260925 09:23:03.543711 16354 leader_election.cc:288] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:23:03.543895 16358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:23:03.545338 16358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [term 2 LEADER]: Becoming Leader. State: Replica: 51b955921daa4e9fa596b31f257d7a76, State: Running, Role: LEADER
I20260925 09:23:03.546185 16354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:03.546033 16358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } }
I20260925 09:23:03.549070 16359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "51b955921daa4e9fa596b31f257d7a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } } }
I20260925 09:23:03.549033 16360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51b955921daa4e9fa596b31f257d7a76. Latest consensus state: current_term: 2 leader_uuid: "51b955921daa4e9fa596b31f257d7a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b955921daa4e9fa596b31f257d7a76" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42259 } } }
I20260925 09:23:03.549481 16359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:03.549551 16360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:03.555548 16365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:03.562155 16365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:03.563011 16354 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 09:23:03.572755 16365 catalog_manager.cc:1259] Loaded cluster ID: 9e92e16253e14b1ba9f17014097425e5
I20260925 09:23:03.573177 16365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:23:03.578944 16365 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:23:03.587198 16365 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Loaded TSK: 0
I20260925 09:23:03.589435 16365 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:23:03.947232 16356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:03.947705 16356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:03.947875 16356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:03.957952 16356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:03.958145 16356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:03.958355 16356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:23:03.971346 16356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:39887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=32961
--tserver_master_addrs=127.3.82.62:42259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:03.973441 16356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:03.975382 16356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:04.023165 16391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:04.023805 16392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:04.024499 16356 server_base.cc:1034] running on GCE node
W20260925 09:23:04.025453 16395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:04.026402 16356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:04.028220 16356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:04.029515 16356 hybrid_clock.cc:648] HybridClock initialized: now 1790328184029439 us; error 68 us; skew 500 ppm
I20260925 09:23:04.029980 16356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:04 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:04.034025 16356 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:23:04.037035 16356 webserver.cc:460] Webserver started at http://127.3.82.1:32961/ using document root <none> and password file <none>
I20260925 09:23:04.038113 16356 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:04.038287 16356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:23:04 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:04.093783 16356 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:23:04.107898 16402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:04.110651 16356 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.006s sys 0.007s
I20260925 09:23:04.110971 16356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "75e727d255ed4379bcdcbb12329cf88e"
format_stamp: "Formatted at 2026-09-25 09:23:00 on dist-test-slave-4wm5"
server_key: "acd6e4e343eab74bd636898d9033d0ed"
server_key_iv: "4aca4015e1e22970e10a64c15e2b8ace"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:04.112498 16356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:04.162689 16356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:04.171171 16356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:04.171559 16356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:04.194308 16356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:04.196571 16356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:04.196800 16356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 09:23:04.197031 16356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:04.197160 16356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:04.259346 16356 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:39887
I20260925 09:23:04.259431 16515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:39887 every 8 connection(s)
I20260925 09:23:04.261323 16356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:23:04.262424 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16356
I20260925 09:23:04.263247 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:45259
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:42259
--webserver_port=38379
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:23:04 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:04.328233 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:04.285105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42259 (local address 127.3.82.1:54299)
0925 09:23:04.299416 (+ 14311us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:04.299486 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:04.303602 (+ 4116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:04.306077 (+ 2475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:04.306088 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:04.306724 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:04.307482 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:04.307506 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:04.310010 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:04.310016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:04.312175 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:04.312194 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:04.314078 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:04.314086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:04.314363 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:04.315189 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:04.315228 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:04.318127 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:04.322904 (+ 4777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:04.322913 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:04.322927 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:04.323568 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:04.324173 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:04.324178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:04.324181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:04.324441 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:04.325494 (+ 1053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:04.325502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:04.325907 (+ 405us) client_negotiation.cc:769] Sending connection context
0925 09:23:04.326360 (+ 453us) client_negotiation.cc:240] Negotiation successful
0925 09:23:04.326946 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13937,"thread_start_us":13669,"threads_started":1}
I20260925 09:23:04.331202 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:04.286233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:54299 (local address 127.3.82.62:42259)
0925 09:23:04.302686 (+ 16453us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:04.302712 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:04.302774 (+ 62us) server_negotiation.cc:407] Connection header received
0925 09:23:04.304402 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:04.304422 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:04.304910 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:04.305344 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:04.307763 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:04.309752 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:04.312372 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:04.313447 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:04.313636 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:04.318493 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:04.318506 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:04.318526 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:04.318576 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:04.322686 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:04.323769 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:04.323778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:04.323786 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:04.324022 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:04.324584 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:04.324589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:04.324592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:04.325227 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:04.325417 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:04.329046 (+ 3629us) server_negotiation.cc:299] Negotiation successful
0925 09:23:04.329954 (+ 908us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16106,"thread_start_us":16447,"threads_started":1}
I20260925 09:23:04.335443 16516 heartbeater.cc:347] Connected to a master server at 127.3.82.62:42259
I20260925 09:23:04.335819 16516 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:04.336781 16516 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
W20260925 09:23:04.340678 16516 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:42259 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:23:04.340972 16516 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:04.341298 16516 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
I20260925 09:23:04.343029 16516 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:04.343453 16516 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
W20260925 09:23:04.345103 16516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:23:04.732522 16517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:04.733055 16517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:04.733286 16517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:04.745374 16517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:04.745630 16517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:04.745891 16517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:23:04.762279 16517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:45259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=38379
--tserver_master_addrs=127.3.82.62:42259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:04.764429 16517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:04.766417 16517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:04.792068 16536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:04.792214 16537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:04.799989 16539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:04.801020 16517 server_base.cc:1034] running on GCE node
I20260925 09:23:04.801853 16517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:04.803313 16517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:04.804641 16517 hybrid_clock.cc:648] HybridClock initialized: now 1790328184804567 us; error 65 us; skew 500 ppm
I20260925 09:23:04.805114 16517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:04 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:04.809728 16517 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:23:04.812014 16517 webserver.cc:460] Webserver started at http://127.3.82.2:38379/ using document root <none> and password file <none>
I20260925 09:23:04.813084 16517 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:04.813253 16517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:23:04 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:04.846768 16517 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.010s sys 0.000s
Sep 25 09:23:04 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:04.858280 16547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:04.860446 16517 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.006s sys 0.000s
I20260925 09:23:04.860738 16517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "955274ee3d694643b6ce9743b9cbe09d"
format_stamp: "Formatted at 2026-09-25 09:23:01 on dist-test-slave-4wm5"
server_key: "6323e147c9e34100d093e2fcafe72f8b"
server_key_iv: "b6921df7f916341074f72f9f8e0e574e"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:04.862414 16517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:04.935415 16517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:04.940300 16517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:04.940625 16517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:04.941948 16517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:04.943831 16517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:04.943961 16517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:04.944113 16517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:04.944203 16517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:05.089708 16517 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:45259
I20260925 09:23:05.089783 16660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:45259 every 8 connection(s)
I20260925 09:23:05.091917 16517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:23:05.099050 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16517
I20260925 09:23:05.099831 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.82.20:55057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:41063
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:42259
--webserver_port=34563
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:23:05 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:05.146128 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:05.105889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:40737 (local address 127.3.82.62:42259)
0925 09:23:05.118328 (+ 12439us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:05.118336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:05.119479 (+ 1143us) server_negotiation.cc:407] Connection header received
0925 09:23:05.122559 (+ 3080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:05.122567 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:05.122689 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:05.122784 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:05.125351 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:05.126634 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:05.129786 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:05.130500 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:05.130629 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:05.135658 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:05.135666 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:05.135671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:05.135721 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:05.139829 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:05.141312 (+ 1483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:05.141319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:05.141323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:05.141617 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:05.142666 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:05.142671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:05.142676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:05.143291 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:05.143521 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:05.145236 (+ 1715us) server_negotiation.cc:299] Negotiation successful
0925 09:23:05.145611 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12275,"thread_start_us":12156,"threads_started":1}
I20260925 09:23:05.149893 16663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:05.105220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42259 (local address 127.3.82.2:40737)
0925 09:23:05.119207 (+ 13987us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:05.119296 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:05.122142 (+ 2846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:05.123437 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:05.123449 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:05.124149 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:05.125100 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:05.125131 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:05.126848 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:05.126853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:05.129566 (+ 2713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:05.129595 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:05.131049 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:05.131056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:05.131369 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:05.132280 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:05.132314 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:05.135281 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:05.140197 (+ 4916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:05.140213 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:05.140241 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:05.141109 (+ 868us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:05.141905 (+ 796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:05.141910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:05.141914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:05.142420 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:05.143706 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:05.143716 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:05.144475 (+ 759us) client_negotiation.cc:769] Sending connection context
0925 09:23:05.146728 (+ 2253us) client_negotiation.cc:240] Negotiation successful
0925 09:23:05.147726 (+ 998us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13491,"thread_start_us":14204,"threads_started":1}
I20260925 09:23:05.154862 16661 heartbeater.cc:347] Connected to a master server at 127.3.82.62:42259
I20260925 09:23:05.155259 16661 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:05.156260 16661 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
I20260925 09:23:05.158782 16319 ts_manager.cc:194] Registered new tserver with Master: 955274ee3d694643b6ce9743b9cbe09d (127.3.82.2:45259)
I20260925 09:23:05.161067 16319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:40737
I20260925 09:23:05.345371 16516 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:05.345955 16516 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
I20260925 09:23:05.347352 16319 ts_manager.cc:194] Registered new tserver with Master: 75e727d255ed4379bcdcbb12329cf88e (127.3.82.1:39887)
I20260925 09:23:05.348701 16319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:54299
Sep 25 09:23:05 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
W20260925 09:23:05.640559 16665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:05.640990 16665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:05.641149 16665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:05.652700 16665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:05.652854 16665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:05.653033 16665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:23:05.668588 16665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.82.20:55057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:41063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=34563
--tserver_master_addrs=127.3.82.62:42259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:05.670321 16665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:05.672017 16665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:05.689713 16671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:05.690356 16670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:05.694317 16665 server_base.cc:1034] running on GCE node
W20260925 09:23:05.694665 16673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:05.695514 16665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:05.696797 16665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:05.698036 16665 hybrid_clock.cc:648] HybridClock initialized: now 1790328185697972 us; error 53 us; skew 500 ppm
I20260925 09:23:05.698451 16665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:05 dist-test-slave-4wm5 krb5kdc[15307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:05.702797 16665 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:23:05.704871 16665 webserver.cc:460] Webserver started at http://127.3.82.3:34563/ using document root <none> and password file <none>
I20260925 09:23:05.705924 16665 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:05.706095 16665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 09:23:05 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:05.733328 16665 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.007s
I20260925 09:23:05.746343 16681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:05.748922 16665 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.000s sys 0.007s
I20260925 09:23:05.749169 16665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "898f465898544e9496ce3ff02eb96b82"
format_stamp: "Formatted at 2026-09-25 09:23:02 on dist-test-slave-4wm5"
server_key: "5489d4a9255c4a8813aa29f63612f962"
server_key_iv: "5135ac8fcc19bdcd8c795c6ec2dd5fb5"
server_key_version: "kuduclusterkey@0"
I20260925 09:23:05.750722 16665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 25 09:23:05 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for HTTP/127.3.82.20@KRBTEST.COM
I20260925 09:23:05.789366 16665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:05.794963 16665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:05.795296 16665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:05.796777 16665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:05.811403 16665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:05.811565 16665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:05.811676 16665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:05.811745 16665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:05.999572 16665 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:41063
I20260925 09:23:05.999617 16794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:41063 every 8 connection(s)
I20260925 09:23:06.001242 16665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:23:06.011632 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16665
Sep 25 09:23:06 dist-test-slave-4wm5 krb5kdc[15307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:06.035264 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.013468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.3.82.62:42259)
0925 09:23:06.015932 (+ 2464us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.015937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.015953 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:23:06.016024 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.016029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.016090 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.016187 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:06.018816 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.020021 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.022057 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.022697 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.022810 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.027219 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:06.027227 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:06.027231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:06.027288 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:06.031155 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.032200 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.032207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.032212 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.032376 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.032855 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.032861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.032865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.033277 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:06.033581 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.034575 (+ 994us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.034839 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2337,"thread_start_us":2207,"threads_started":1}
I20260925 09:23:06.041850 16798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.006519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:52311 (local address 127.3.82.62:42259)
0925 09:23:06.014864 (+ 8345us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.014873 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.014904 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:23:06.015009 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.015017 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.015128 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.015304 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:06.018553 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.019918 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.022593 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.023282 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.023398 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.033283 (+ 9885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:06.033288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:06.033292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:06.033333 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:06.037470 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.038548 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.038554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.038558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.038712 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.039342 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.039350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.039356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.039775 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:06.039916 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.041098 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.041357 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8161,"thread_start_us":8007,"threads_started":1}
I20260925 09:23:06.043020 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.005633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42259 (local address 127.3.82.3:52311)
0925 09:23:06.008300 (+ 2667us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.008364 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:06.011569 (+ 3205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:06.016297 (+ 4728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:06.016313 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:06.017096 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:06.018178 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.018257 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.020133 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.020139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.022370 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.022390 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.023735 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.023744 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.024100 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.029923 (+ 5823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:06.029962 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:06.032961 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:06.037641 (+ 4680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:06.037652 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:06.037673 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:06.038389 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:06.038858 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:06.038862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:06.038866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:06.039196 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:06.040012 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:06.040021 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:06.040560 (+ 539us) client_negotiation.cc:769] Sending connection context
0925 09:23:06.041001 (+ 441us) client_negotiation.cc:240] Negotiation successful
0925 09:23:06.041618 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2331,"thread_start_us":2004,"threads_started":1}
I20260925 09:23:06.044641 16795 heartbeater.cc:347] Connected to a master server at 127.3.82.62:42259
I20260925 09:23:06.044942 16795 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:06.045687 16795 heartbeater.cc:511] Master 127.3.82.62:42259 requested a full tablet report, sending...
I20260925 09:23:06.047521 16319 ts_manager.cc:194] Registered new tserver with Master: 898f465898544e9496ce3ff02eb96b82 (127.3.82.3:41063)
I20260925 09:23:06.048463 16319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:52311
I20260925 09:23:06.051746 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:23:06.075536 16798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.055219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.3.82.62:42259)
0925 09:23:06.055415 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.055421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.058645 (+ 3224us) server_negotiation.cc:407] Connection header received
0925 09:23:06.058871 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.058880 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.059010 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.059124 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:06.061155 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.062304 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.064240 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.064668 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.064781 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.068411 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:06.068417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:06.068421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:06.068526 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:06.071909 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.072975 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.072981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.072983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.073173 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.073675 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.073680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.073682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.074164 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:06.074433 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.074902 (+ 469us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.075119 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 09:23:06.082970 16319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:23:06.091729 16319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:23:06.132256 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.108903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60310)
0925 09:23:06.110053 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.110122 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:06.110383 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:06.123207 (+ 12824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:06.123213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:06.123244 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:06.124001 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.124019 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.126327 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.126334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.128765 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.128780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.131045 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.131052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.131304 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.131376 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 09:23:06.131531 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 09:23:06.131779 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":80,"spinlock_wait_cycles":8960,"thread_start_us":703,"threads_started":1}
I20260925 09:23:06.133038 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.109395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.3.82.1:39887)
0925 09:23:06.122584 (+ 13189us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.122591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.122621 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:23:06.122725 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.122734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.122958 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.124226 (+ 1268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:06.124481 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.126157 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.129002 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.130759 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.130902 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.131148 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.132591 (+ 1443us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.132774 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13036,"thread_start_us":12728,"threads_started":1}
I20260925 09:23:06.134827 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.111796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:45259 (local address 127.0.0.1:56860)
0925 09:23:06.124306 (+ 12510us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.124328 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:06.124458 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:06.125189 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:06.125194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:06.125220 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:06.125668 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.125687 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.129683 (+ 3996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.129690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.131877 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.131892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.133789 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.133794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.134011 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.134041 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:23:06.134162 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 09:23:06.134336 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12371,"mutex_wait_us":2,"thread_start_us":8923,"threads_started":1}
I20260925 09:23:06.135015 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.111972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.3.82.2:45259)
0925 09:23:06.122122 (+ 10150us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.122131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.124611 (+ 2480us) server_negotiation.cc:407] Connection header received
0925 09:23:06.124729 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.124740 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.124963 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.125189 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:06.125980 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.129517 (+ 3537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.132106 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.133560 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.133697 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.133959 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.134538 (+ 579us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.134704 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9959,"thread_start_us":9830,"threads_started":1}
I20260925 09:23:06.135462 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.109595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.3.82.3:41063)
0925 09:23:06.121663 (+ 12068us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.121678 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.121726 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:23:06.121875 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.121884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.122275 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.122587 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:06.126076 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.127736 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.131250 (+ 3514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.134033 (+ 2783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.134178 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.134403 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.134940 (+ 537us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.135149 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11862,"thread_start_us":11688,"threads_started":1}
I20260925 09:23:06.135576 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.108986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:41063 (local address 127.0.0.1:46014)
0925 09:23:06.121113 (+ 12127us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.121140 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:06.121327 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:06.123110 (+ 1783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:06.123125 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:06.123181 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:06.125790 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.125806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.127938 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.127944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.130466 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.130480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.134276 (+ 3796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.134281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.134441 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.134477 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:23:06.134573 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 09:23:06.134720 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11925,"mutex_wait_us":1757,"thread_start_us":14995,"threads_started":1}
I20260925 09:23:06.137570 16451 tablet_service.cc:1467] Processing CreateTablet for tablet dc18a3fd38004c52a7bbf887cee160dd (DEFAULT_TABLE table=test-table [id=d079c42922a749b899a9a595268067ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:06.138959 16451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc18a3fd38004c52a7bbf887cee160dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:06.139837 16730 tablet_service.cc:1467] Processing CreateTablet for tablet dc18a3fd38004c52a7bbf887cee160dd (DEFAULT_TABLE table=test-table [id=d079c42922a749b899a9a595268067ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:06.139837 16596 tablet_service.cc:1467] Processing CreateTablet for tablet dc18a3fd38004c52a7bbf887cee160dd (DEFAULT_TABLE table=test-table [id=d079c42922a749b899a9a595268067ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:06.141649 16730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc18a3fd38004c52a7bbf887cee160dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:06.141649 16596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc18a3fd38004c52a7bbf887cee160dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:06.152844 16823 tablet_bootstrap.cc:492] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Bootstrap starting.
I20260925 09:23:06.156924 16823 tablet_bootstrap.cc:654] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:06.158355 16823 log.cc:824] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:06.159902 16824 tablet_bootstrap.cc:492] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Bootstrap starting.
I20260925 09:23:06.161202 16825 tablet_bootstrap.cc:492] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Bootstrap starting.
I20260925 09:23:06.162042 16823 tablet_bootstrap.cc:492] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: No bootstrap required, opened a new log
I20260925 09:23:06.162319 16823 ts_tablet_manager.cc:1397] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260925 09:23:06.165207 16825 tablet_bootstrap.cc:654] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:06.166044 16824 tablet_bootstrap.cc:654] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:06.166671 16661 heartbeater.cc:503] Master 127.3.82.62:42259 was elected leader, sending a full tablet report...
I20260925 09:23:06.166707 16825 log.cc:824] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:06.167462 16824 log.cc:824] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:06.170712 16825 tablet_bootstrap.cc:492] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: No bootstrap required, opened a new log
I20260925 09:23:06.171033 16825 ts_tablet_manager.cc:1397] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260925 09:23:06.174682 16823 raft_consensus.cc:348] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.175217 16823 raft_consensus.cc:374] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:06.175323 16823 raft_consensus.cc:729] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75e727d255ed4379bcdcbb12329cf88e, State: Initialized, Role: FOLLOWER
I20260925 09:23:06.176038 16823 consensus_queue.cc:260] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.177088 16824 tablet_bootstrap.cc:492] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: No bootstrap required, opened a new log
I20260925 09:23:06.177416 16824 ts_tablet_manager.cc:1397] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.007s
I20260925 09:23:06.178618 16516 heartbeater.cc:503] Master 127.3.82.62:42259 was elected leader, sending a full tablet report...
I20260925 09:23:06.179340 16823 ts_tablet_manager.cc:1428] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260925 09:23:06.183507 16825 raft_consensus.cc:348] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.183947 16825 raft_consensus.cc:374] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:06.184056 16825 raft_consensus.cc:729] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 898f465898544e9496ce3ff02eb96b82, State: Initialized, Role: FOLLOWER
I20260925 09:23:06.184756 16825 consensus_queue.cc:260] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.186025 16795 heartbeater.cc:503] Master 127.3.82.62:42259 was elected leader, sending a full tablet report...
I20260925 09:23:06.186733 16825 ts_tablet_manager.cc:1428] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260925 09:23:06.186903 16824 raft_consensus.cc:348] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.187408 16824 raft_consensus.cc:374] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:06.187507 16824 raft_consensus.cc:729] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955274ee3d694643b6ce9743b9cbe09d, State: Initialized, Role: FOLLOWER
I20260925 09:23:06.188220 16824 consensus_queue.cc:260] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.190160 16824 ts_tablet_manager.cc:1428] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
W20260925 09:23:06.263623 16796 tablet.cc:2369] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:23:06.312381 16520 tablet.cc:2369] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:06.336899 16831 raft_consensus.cc:482] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:23:06.337291 16831 raft_consensus.cc:504] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.339177 16831 leader_election.cc:288] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75e727d255ed4379bcdcbb12329cf88e (127.3.82.1:39887), 898f465898544e9496ce3ff02eb96b82 (127.3.82.3:41063)
I20260925 09:23:06.349681 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.339752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:46861 (local address 127.3.82.3:41063)
0925 09:23:06.340001 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.340007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.340892 (+ 885us) server_negotiation.cc:407] Connection header received
0925 09:23:06.341073 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.341080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.341192 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.341285 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:06.342447 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.343748 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.347059 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.348451 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.348599 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.348722 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.349264 (+ 542us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.349405 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:23:06.349720 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.340117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:48035 (local address 127.3.82.1:39887)
0925 09:23:06.340341 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.340349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.340806 (+ 457us) server_negotiation.cc:407] Connection header received
0925 09:23:06.341049 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.341055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.341164 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.341285 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:06.342294 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.343612 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.347057 (+ 3445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.348425 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.348610 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.348722 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.349314 (+ 592us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.349449 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:23:06.350256 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.339626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.3.82.2:48035)
0925 09:23:06.340659 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.340681 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:06.340880 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:06.341362 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:06.341367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:06.341483 (+ 116us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:06.342042 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.342061 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.343855 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.343863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.346823 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.346845 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.348820 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.348828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.349134 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.349178 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 09:23:06.349258 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 09:23:06.349399 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":195,"spinlock_wait_cycles":47360,"thread_start_us":863,"threads_started":1}
I20260925 09:23:06.350297 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.339673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:41063 (local address 127.3.82.2:46861)
0925 09:23:06.340773 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.340789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:06.340884 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:06.341362 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:06.341367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:06.341410 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:06.342268 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.342277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.343902 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.343910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.346824 (+ 2914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:06.346844 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.348834 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.348840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:06.349025 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.349068 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 09:23:06.349193 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 09:23:06.349385 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":227,"thread_start_us":616,"threads_started":1}
I20260925 09:23:06.351936 16750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc18a3fd38004c52a7bbf887cee160dd" candidate_uuid: "955274ee3d694643b6ce9743b9cbe09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "898f465898544e9496ce3ff02eb96b82" is_pre_election: true
I20260925 09:23:06.351938 16471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc18a3fd38004c52a7bbf887cee160dd" candidate_uuid: "955274ee3d694643b6ce9743b9cbe09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75e727d255ed4379bcdcbb12329cf88e" is_pre_election: true
I20260925 09:23:06.352610 16750 raft_consensus.cc:2454] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955274ee3d694643b6ce9743b9cbe09d in term 0.
I20260925 09:23:06.352603 16471 raft_consensus.cc:2454] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 955274ee3d694643b6ce9743b9cbe09d in term 0.
I20260925 09:23:06.353596 16551 leader_election.cc:302] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898f465898544e9496ce3ff02eb96b82, 955274ee3d694643b6ce9743b9cbe09d; no voters:
I20260925 09:23:06.354168 16831 raft_consensus.cc:2781] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:23:06.354347 16831 raft_consensus.cc:482] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:23:06.354461 16831 raft_consensus.cc:3037] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:06.357128 16831 raft_consensus.cc:504] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.358002 16831 leader_election.cc:288] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [CANDIDATE]: Term 1 election: Requested vote from peers 75e727d255ed4379bcdcbb12329cf88e (127.3.82.1:39887), 898f465898544e9496ce3ff02eb96b82 (127.3.82.3:41063)
I20260925 09:23:06.358747 16750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc18a3fd38004c52a7bbf887cee160dd" candidate_uuid: "955274ee3d694643b6ce9743b9cbe09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "898f465898544e9496ce3ff02eb96b82"
I20260925 09:23:06.358772 16471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc18a3fd38004c52a7bbf887cee160dd" candidate_uuid: "955274ee3d694643b6ce9743b9cbe09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75e727d255ed4379bcdcbb12329cf88e"
I20260925 09:23:06.358980 16750 raft_consensus.cc:3037] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:06.359189 16471 raft_consensus.cc:3037] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:23:06.360018 16662 tablet.cc:2369] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:06.361905 16750 raft_consensus.cc:2454] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955274ee3d694643b6ce9743b9cbe09d in term 1.
I20260925 09:23:06.362092 16471 raft_consensus.cc:2454] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 955274ee3d694643b6ce9743b9cbe09d in term 1.
I20260925 09:23:06.362478 16551 leader_election.cc:302] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898f465898544e9496ce3ff02eb96b82, 955274ee3d694643b6ce9743b9cbe09d; no voters:
I20260925 09:23:06.362890 16831 raft_consensus.cc:2781] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:06.363437 16831 raft_consensus.cc:686] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 1 LEADER]: Becoming Leader. State: Replica: 955274ee3d694643b6ce9743b9cbe09d, State: Running, Role: LEADER
I20260925 09:23:06.363940 16831 consensus_queue.cc:237] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } }
I20260925 09:23:06.369408 16318 catalog_manager.cc:5495] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d reported cstate change: term changed from 0 to 1, leader changed from <none> to 955274ee3d694643b6ce9743b9cbe09d (127.3.82.2). New cstate: current_term: 1 leader_uuid: "955274ee3d694643b6ce9743b9cbe09d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "955274ee3d694643b6ce9743b9cbe09d" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45259 } health_report { overall_health: HEALTHY } } }
I20260925 09:23:06.437573 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.431857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.3.82.2:45259)
0925 09:23:06.432103 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.432111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.432139 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:23:06.432287 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.432291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.432380 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.432552 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:23:06.432990 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.433791 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.435222 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.435666 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.435770 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.436329 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:23:06.436889 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:23:06.437004 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.437122 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.437195 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:23:06.446666 16471 raft_consensus.cc:1261] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 1 FOLLOWER]: Refusing update from remote peer 955274ee3d694643b6ce9743b9cbe09d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:06.446708 16750 raft_consensus.cc:1261] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 1 FOLLOWER]: Refusing update from remote peer 955274ee3d694643b6ce9743b9cbe09d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:06.447903 16834 consensus_queue.cc:1035] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [LEADER]: Connected to new peer: Peer: permanent_uuid: "898f465898544e9496ce3ff02eb96b82" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 41063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:06.448333 16831 consensus_queue.cc:1035] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [LEADER]: Connected to new peer: Peer: permanent_uuid: "75e727d255ed4379bcdcbb12329cf88e" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 39887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:06.462468 16838 mvcc.cc:205] Tried to move back new op lower bound from 7333184251674816512 to 7333184251351392256. Current Snapshot: MvccSnapshot[applied={T|T < 7333184251674816512}]
I20260925 09:23:06.463398 16839 mvcc.cc:205] Tried to move back new op lower bound from 7333184251674816512 to 7333184251351392256. Current Snapshot: MvccSnapshot[applied={T|T < 7333184251674816512}]
I20260925 09:23:06.476229 16319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:06.476559 16319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:06.481261 16319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51b955921daa4e9fa596b31f257d7a76: Sending DeleteTablet for 3 replicas of tablet dc18a3fd38004c52a7bbf887cee160dd
I20260925 09:23:06.482668 16451 tablet_service.cc:1514] Processing DeleteTablet for tablet dc18a3fd38004c52a7bbf887cee160dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:06 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:60310
I20260925 09:23:06.482666 16730 tablet_service.cc:1514] Processing DeleteTablet for tablet dc18a3fd38004c52a7bbf887cee160dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:06 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:46014
I20260925 09:23:06.482875 16596 tablet_service.cc:1514] Processing DeleteTablet for tablet dc18a3fd38004c52a7bbf887cee160dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:06 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:56860
I20260925 09:23:06.488737 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16356
I20260925 09:23:06.505162 16848 tablet_replica.cc:330] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: stopping tablet replica
I20260925 09:23:06.505812 16847 tablet_replica.cc:330] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: stopping tablet replica
I20260925 09:23:06.506390 16846 tablet_replica.cc:330] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: stopping tablet replica
I20260925 09:23:06.506913 16848 raft_consensus.cc:2229] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:23:06.507177 16846 raft_consensus.cc:2229] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:06.507275 16847 raft_consensus.cc:2229] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:06.507769 16846 raft_consensus.cc:2258] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:06.508278 16847 raft_consensus.cc:2258] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:06.508826 16848 raft_consensus.cc:2258] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:06.512332 16848 ts_tablet_manager.cc:1905] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:06.512544 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.489656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.3.82.1:39887)
0925 09:23:06.489882 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.489889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.489980 (+ 91us) server_negotiation.cc:407] Connection header received
0925 09:23:06.490193 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.490198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.490282 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.490387 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:06.496365 (+ 5978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.497950 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.499872 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.500812 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.501003 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.504534 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:06.504559 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:06.504587 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:06.504641 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:06.508917 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.509927 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.509936 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.509940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.510135 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.510721 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.510726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.510729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.511195 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:06.511504 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.511862 (+ 358us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.512100 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:23:06.512853 16846 ts_tablet_manager.cc:1905] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:06.513491 16511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:06.522284 16846 ts_tablet_manager.cc:1918] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:06.522547 16846 log.cc:1196] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/dc18a3fd38004c52a7bbf887cee160dd
I20260925 09:23:06.523343 16846 ts_tablet_manager.cc:1939] T dc18a3fd38004c52a7bbf887cee160dd P 898f465898544e9496ce3ff02eb96b82: Deleting consensus metadata
I20260925 09:23:06.525695 16848 ts_tablet_manager.cc:1918] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:06.525955 16306 catalog_manager.cc:4841] TS 898f465898544e9496ce3ff02eb96b82 (127.3.82.3:41063): tablet dc18a3fd38004c52a7bbf887cee160dd (table test-table [id=d079c42922a749b899a9a595268067ec]) successfully deleted
I20260925 09:23:06.526620 16848 log.cc:1196] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/dc18a3fd38004c52a7bbf887cee160dd
I20260925 09:23:06.527357 16848 ts_tablet_manager.cc:1939] T dc18a3fd38004c52a7bbf887cee160dd P 955274ee3d694643b6ce9743b9cbe09d: Deleting consensus metadata
I20260925 09:23:06.529542 16305 catalog_manager.cc:4841] TS 955274ee3d694643b6ce9743b9cbe09d (127.3.82.2:45259): tablet dc18a3fd38004c52a7bbf887cee160dd (table test-table [id=d079c42922a749b899a9a595268067ec]) successfully deleted
I20260925 09:23:06.514580 16847 ts_tablet_manager.cc:1905] T dc18a3fd38004c52a7bbf887cee160dd P 75e727d255ed4379bcdcbb12329cf88e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:06.642941 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16356
I20260925 09:23:06.660941 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16517
W20260925 09:23:06.661039 16304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.82.1:39887 (error 108)
I20260925 09:23:06.663635 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.662850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60326)
0925 09:23:06.663355 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.663468 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":272,"threads_started":1}
W20260925 09:23:06.664000 16304 catalog_manager.cc:4584] TS 75e727d255ed4379bcdcbb12329cf88e (127.3.82.1:39887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc18a3fd38004c52a7bbf887cee160dd: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
I20260925 09:23:06.678810 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.662122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.3.82.2:45259)
0925 09:23:06.662319 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.662323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.662341 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:23:06.662400 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.662404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.662468 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.662554 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:06.664420 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.665301 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.667305 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.667841 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.667959 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.671563 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:06.671582 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:06.671601 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:06.671648 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:06.675309 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.676295 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.676303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.676306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.676528 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.676947 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.676950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.676952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.677409 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:06.677683 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.678153 (+ 470us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.678374 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:23:06.679584 16656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:06.717808 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.717295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60328)
0925 09:23:06.717557 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.717663 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260925 09:23:06.719228 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.718732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60342)
0925 09:23:06.718925 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.719086 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 09:23:06.780100 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.779627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60352)
0925 09:23:06.779854 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.779944 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 09:23:06.781222 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.780850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60366)
0925 09:23:06.781006 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.781086 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 09:23:06.797763 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16517
I20260925 09:23:06.816309 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16665
I20260925 09:23:06.831303 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.817081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.3.82.3:41063)
0925 09:23:06.817281 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:06.817288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:06.817390 (+ 102us) server_negotiation.cc:407] Connection header received
0925 09:23:06.817646 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:06.817651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:06.817719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:06.817829 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:06.819225 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.819954 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.821213 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:06.821617 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:06.821752 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:06.824625 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:06.824642 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:06.824659 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:06.824707 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:06.827844 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.828687 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.828696 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.828698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.828841 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:06.829244 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:06.829250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:06.829253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:06.829642 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:06.829991 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:06.830675 (+ 684us) server_negotiation.cc:299] Negotiation successful
0925 09:23:06.830901 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:23:06.832015 16790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:06.862167 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.861574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60372)
0925 09:23:06.861895 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.862018 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260925 09:23:06.863442 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:06.862963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60378)
0925 09:23:06.863172 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:06.863254 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260925 09:23:06.925253 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16665
I20260925 09:23:06.942571 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16286
I20260925 09:23:06.943931 16349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:07.061487 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:07.060795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:39887 (local address 127.0.0.1:60388)
0925 09:23:07.061051 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:07.061232 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260925 09:23:07.062043 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16286
I20260925 09:23:07.089782 3400 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 09:23:07.426584 3400 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 09:23:07.426834 3400 mini_ranger.cc:66] Stopping Ranger...
I20260925 09:23:07.774051 3400 mini_ranger.cc:68] Stopped Ranger
2026-09-25 09:23:07.774 UTC [15337] LOG: received smart shutdown request
2026-09-25 09:23:07.778 UTC [15337] LOG: background worker "logical replication launcher" (PID 15346) exited with exit code 1
2026-09-25 09:23:07.778 UTC [15341] LOG: shutting down
2026-09-25 09:23:07.983 UTC [15337] LOG: database system is shut down
2026-09-25T09:23:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56786 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:23:08 dist-test-slave-4wm5 krb5kdc[16863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:23:08 dist-test-slave-4wm5 krb5kdc[16863](info): set up 2 sockets
Sep 25 09:23:08 dist-test-slave-4wm5 krb5kdc[16863](info): commencing operation
krb5kdc: starting...
W20260925 09:23:09.266480 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:23:09 dist-test-slave-4wm5 krb5kdc[16863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:23:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:23:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:23:09.523087 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42129
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42129,127.3.82.61:45823,127.3.82.60:46123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:09.870450 16876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:09.870841 16876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:09.871016 16876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:09.878641 16876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:09.878897 16876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:09.879029 16876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:09.879140 16876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:09.879220 16876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:09.889676 16876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42129,127.3.82.61:45823,127.3.82.60:46123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:09.891211 16876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:09.892632 16876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:09.906225 16881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:09.906479 16882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:09.906354 16884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:09.907403 16876 server_base.cc:1034] running on GCE node
I20260925 09:23:09.908308 16876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:09.910017 16876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:09.911278 16876 hybrid_clock.cc:648] HybridClock initialized: now 1790328189911238 us; error 40 us; skew 500 ppm
I20260925 09:23:09.911667 16876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:09 dist-test-slave-4wm5 krb5kdc[16863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:09.920774 16876 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:09.922816 16876 webserver.cc:460] Webserver started at http://127.3.82.62:45743/ using document root <none> and password file <none>
I20260925 09:23:09.923635 16876 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:09.923799 16876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:09.924144 16876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:09.927510 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "b55936e1289548fc870e990cc7a23ddf"
format_stamp: "Formatted at 2026-09-25 09:23:09 on dist-test-slave-4wm5"
server_key: "c31aa4d2f21d5cfc6fc2051a4c05d350"
server_key_iv: "1d01340cdf1f467dbf4ca13d3c447add"
server_key_version: "encryptionkey@0"
I20260925 09:23:09.928424 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "b55936e1289548fc870e990cc7a23ddf"
format_stamp: "Formatted at 2026-09-25 09:23:09 on dist-test-slave-4wm5"
server_key: "c31aa4d2f21d5cfc6fc2051a4c05d350"
server_key_iv: "1d01340cdf1f467dbf4ca13d3c447add"
server_key_version: "encryptionkey@0"
I20260925 09:23:09.934274 16876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260925 09:23:09.938205 16891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:09.939872 16876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:23:09.940084 16876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "b55936e1289548fc870e990cc7a23ddf"
format_stamp: "Formatted at 2026-09-25 09:23:09 on dist-test-slave-4wm5"
server_key: "c31aa4d2f21d5cfc6fc2051a4c05d350"
server_key_iv: "1d01340cdf1f467dbf4ca13d3c447add"
server_key_version: "encryptionkey@0"
I20260925 09:23:09.940322 16876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:09.969015 16876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:09.974579 16876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:09.974978 16876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:09.998805 16876 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:42129
I20260925 09:23:09.998840 16942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:42129 every 8 connection(s)
I20260925 09:23:10.000538 16876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:10.005041 16943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:10.008646 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16876
I20260925 09:23:10.008924 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:23:10.009648 3400 external_mini_cluster.cc:1428] Setting key e9308ef8d83776d645e82f30662ff97a
I20260925 09:23:10.013016 16943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:23:10 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.61@KRBTEST.COM" created.
I20260925 09:23:10.042853 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.016998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.3.82.62:42129)
0925 09:23:10.018148 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:10.018169 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:10.018803 (+ 634us) server_negotiation.cc:407] Connection header received
0925 09:23:10.022104 (+ 3301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:10.022111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:10.022404 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:10.022648 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:10.024502 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.025813 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.028389 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.029185 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.029350 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.033991 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:10.034025 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:10.034041 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:10.034079 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:10.037670 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.038523 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.038542 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.038545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.038714 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.039175 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.039188 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.039191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.039779 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:10.040024 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:10.040671 (+ 647us) server_negotiation.cc:299] Negotiation successful
0925 09:23:10.041118 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":520,"threads_started":1}
I20260925 09:23:10.042876 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.016997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42129 (local address 127.0.0.1:40990)
0925 09:23:10.018575 (+ 1578us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.018633 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:10.021596 (+ 2963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:10.022697 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:10.022707 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:10.023620 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:10.024250 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.024281 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.026050 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.026058 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.028144 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.028164 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.029469 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.029493 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.029721 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.030508 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:10.030548 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:10.033675 (+ 3127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:10.037882 (+ 4207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.037895 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.037916 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.038357 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.038854 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.038859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.038864 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.039044 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.039938 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:10.039950 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:10.040235 (+ 285us) client_negotiation.cc:769] Sending connection context
0925 09:23:10.040517 (+ 282us) client_negotiation.cc:240] Negotiation successful
0925 09:23:10.041253 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":220,"thread_start_us":1012,"threads_started":1}
I20260925 09:23:10.046674 16943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:23:10.047870 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.047425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36554)
0925 09:23:10.047620 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.047743 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260925 09:23:10.048171 16895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
I20260925 09:23:10.050050 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.049435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36560)
0925 09:23:10.049638 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.049769 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 09:23:10.050264 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:45823: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab.
Entry for principal kudu/127.3.82.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab.
I20260925 09:23:10.069747 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:45823
--webserver_interface=127.3.82.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab
--principal=kudu/127.3.82.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42129,127.3.82.61:45823,127.3.82.60:46123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:10.099687 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } attempt: 1
I20260925 09:23:10.100975 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.100421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36564)
0925 09:23:10.100659 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.100789 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 09:23:10.102211 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.101769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36578)
0925 09:23:10.101957 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.102044 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260925 09:23:10.102424 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:45823: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
I20260925 09:23:10.170723 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } attempt: 2
I20260925 09:23:10.172122 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.171540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36580)
0925 09:23:10.171823 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.171944 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260925 09:23:10.173309 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.172899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36582)
0925 09:23:10.173055 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.173153 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:23:10.173570 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:45823: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
I20260925 09:23:10.264851 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } attempt: 3
I20260925 09:23:10.266148 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.265534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36588)
0925 09:23:10.265809 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.265962 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 09:23:10.267374 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.266945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36600)
0925 09:23:10.267111 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.267200 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 09:23:10.267683 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:45823: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
I20260925 09:23:10.410998 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } attempt: 4
I20260925 09:23:10.412460 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.411842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36612)
0925 09:23:10.412219 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.412325 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260925 09:23:10.413720 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.413293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36626)
0925 09:23:10.413465 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.413557 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260925 09:23:10.413913 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:45823: Network error: Client connection negotiation failed: client connection to 127.3.82.61:45823: connect: Connection refused (error 111)
W20260925 09:23:10.421653 16950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:10.422119 16950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:10.422284 16950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:10.430034 16950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:10.430197 16950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:10.430311 16950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:10.430413 16950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:10.430501 16950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:10.441455 16950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42129,127.3.82.61:45823,127.3.82.60:46123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab
--principal=kudu/127.3.82.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:45823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.82.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:10.443117 16950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:10.444801 16950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:10.458287 16958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:10.458333 16956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:10.458242 16955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:10.460163 16950 server_base.cc:1034] running on GCE node
I20260925 09:23:10.461167 16950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:10.462932 16950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:10.464241 16950 hybrid_clock.cc:648] HybridClock initialized: now 1790328190464183 us; error 46 us; skew 500 ppm
I20260925 09:23:10.464731 16950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:10 dist-test-slave-4wm5 krb5kdc[16863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328190, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:10.468551 16950 init.cc:377] Logged in from keytab as kudu/127.3.82.61@KRBTEST.COM (short username kudu)
I20260925 09:23:10.470573 16950 webserver.cc:460] Webserver started at http://127.3.82.61:46803/ using document root <none> and password file <none>
I20260925 09:23:10.471416 16950 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:10.471585 16950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:10.471937 16950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:10.475389 16950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data/instance:
uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e"
format_stamp: "Formatted at 2026-09-25 09:23:10 on dist-test-slave-4wm5"
server_key: "75cca82d933fa957fab08f55b4794261"
server_key_iv: "f17450e6656a466f1d3e67a1dd6e087a"
server_key_version: "encryptionkey@0"
I20260925 09:23:10.476347 16950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal/instance:
uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e"
format_stamp: "Formatted at 2026-09-25 09:23:10 on dist-test-slave-4wm5"
server_key: "75cca82d933fa957fab08f55b4794261"
server_key_iv: "f17450e6656a466f1d3e67a1dd6e087a"
server_key_version: "encryptionkey@0"
I20260925 09:23:10.482345 16950 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260925 09:23:10.486706 16965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:10.488709 16950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:23:10.488969 16950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal
uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e"
format_stamp: "Formatted at 2026-09-25 09:23:10 on dist-test-slave-4wm5"
server_key: "75cca82d933fa957fab08f55b4794261"
server_key_iv: "f17450e6656a466f1d3e67a1dd6e087a"
server_key_version: "encryptionkey@0"
I20260925 09:23:10.489248 16950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:10.510682 16950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:10.515715 16950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:10.516041 16950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:10.536329 16950 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.61:45823
I20260925 09:23:10.536351 17016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.61:45823 every 8 connection(s)
I20260925 09:23:10.538038 16950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
I20260925 09:23:10.542050 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:10.544693 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 16950
I20260925 09:23:10.544904 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-1/wal/instance
I20260925 09:23:10.545480 3400 external_mini_cluster.cc:1428] Setting key 5fe68207b915837dd09aa57f9e53684b
I20260925 09:23:10.552521 17017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.82.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.60@KRBTEST.COM" created.
Sep 25 09:23:10 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328190, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:10.580667 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.556092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.3.82.62:42129)
0925 09:23:10.556650 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:10.556657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:10.557753 (+ 1096us) server_negotiation.cc:407] Connection header received
0925 09:23:10.559828 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:10.559838 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:10.559958 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:10.560101 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:10.561821 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.563116 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.565103 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.566008 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.566175 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.573799 (+ 7624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:10.573805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:10.573808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:10.573916 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:10.576717 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.577925 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.577930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.577932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.578062 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.578587 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.578590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.578592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.578875 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:10.578979 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:10.579884 (+ 905us) server_negotiation.cc:299] Negotiation successful
0925 09:23:10.580168 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":287,"threads_started":1}
I20260925 09:23:10.581342 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.556669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42129 (local address 127.0.0.1:40998)
0925 09:23:10.557546 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.557601 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:10.559491 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:10.560398 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:10.560409 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:10.561004 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:10.561616 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.561634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.563318 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.563322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.564901 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.564931 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.566281 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.566286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.566487 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.567258 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:10.567285 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:10.573373 (+ 6088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:10.576926 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.576940 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.576965 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.577724 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.578195 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.578203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.578210 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.578476 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.579025 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:10.579038 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:10.579469 (+ 431us) client_negotiation.cc:769] Sending connection context
0925 09:23:10.579729 (+ 260us) client_negotiation.cc:240] Negotiation successful
0925 09:23:10.580158 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":386,"threads_started":1}
I20260925 09:23:10.583417 17017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:23:10 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328190, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:23:10.599479 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.584257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:36632)
0925 09:23:10.584489 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.584509 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:10.584624 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:10.585778 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:10.585783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:10.586397 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:10.586544 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.586553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.587972 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.587978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.588961 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.588987 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.590039 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.590045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.590254 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.590883 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:10.590901 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:10.592985 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:10.596705 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.596713 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.596717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.597075 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.597499 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.597505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.597509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.597651 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.598370 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:10.598375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:10.598554 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 09:23:10.598696 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 09:23:10.599011 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 09:23:10.599522 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.584580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.3.82.61:45823)
0925 09:23:10.585278 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:10.585285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:10.585312 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:23:10.585368 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:10.585375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:10.585614 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:10.585794 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:10.586705 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.587816 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.589121 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.589817 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.590016 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.593201 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:10.593213 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:10.593228 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:10.593304 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:10.596555 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.597209 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.597221 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.597224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.597387 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.597779 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.597783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.597786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.598236 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:10.598470 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:10.598849 (+ 379us) server_negotiation.cc:299] Negotiation successful
0925 09:23:10.599154 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":544,"thread_start_us":389,"threads_started":1}
I20260925 09:23:10.602231 17017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:23:10.603593 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.602971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:33852)
0925 09:23:10.603180 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.603363 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260925 09:23:10.603880 16969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab.
Entry for principal kudu/127.3.82.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab.
I20260925 09:23:10.605698 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:46123
--webserver_interface=127.3.82.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab
--principal=kudu/127.3.82.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42129,127.3.82.61:45823,127.3.82.60:46123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:10.606101 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.605612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:33862)
0925 09:23:10.605790 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.605939 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 09:23:10.606361 17017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.655702 17017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 1
I20260925 09:23:10.657109 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.656527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:33872)
0925 09:23:10.656747 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.656868 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260925 09:23:10.658188 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.657782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:33882)
0925 09:23:10.658009 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.658077 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260925 09:23:10.658404 17017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.713328 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } attempt: 5
Sep 25 09:23:10 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:23:10.726629 17017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 2
I20260925 09:23:10.728015 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.727384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47380)
0925 09:23:10.727684 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.727787 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260925 09:23:10.729166 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.728738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47394)
0925 09:23:10.728903 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.728993 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 09:23:10.729398 17017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.729771 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.714151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:55972)
0925 09:23:10.714378 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.714400 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:10.714532 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:10.714931 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:10.714935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:10.715444 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:10.715638 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.715648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.717143 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.717148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.718246 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:10.718268 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.719287 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.719292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:10.719440 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.720135 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:10.720155 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:10.722423 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:10.726422 (+ 3999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.726427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.726430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.727087 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.727602 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:10.727607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:10.727611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:10.727808 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:10.728722 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:10.728727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:10.728911 (+ 184us) client_negotiation.cc:769] Sending connection context
0925 09:23:10.729090 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 09:23:10.729415 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:23:10.729993 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.714316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.3.82.61:45823)
0925 09:23:10.714501 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:10.714507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:10.714531 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:23:10.714660 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:10.714665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:10.714743 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:10.714857 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:10.715811 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.716962 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.718432 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:10.719125 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:10.719259 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:10.722676 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:10.722683 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:10.722687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:10.722730 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:10.726232 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.727275 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.727280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.727283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.727434 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:10.727939 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:10.727944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:10.727950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:10.728470 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:10.728655 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:10.729211 (+ 556us) server_negotiation.cc:299] Negotiation successful
0925 09:23:10.729690 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 09:23:10.731019 16943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:23:10.732097 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.731692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47406)
0925 09:23:10.731858 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.731945 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 09:23:10.732965 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.732660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47410)
0925 09:23:10.732796 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.732864 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 09:23:10.733198 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.784490 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 1
I20260925 09:23:10.786075 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.785336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47418)
0925 09:23:10.785654 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.785787 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 09:23:10.787266 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.786865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47434)
0925 09:23:10.787029 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.787124 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:23:10.787549 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.820658 17017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 3
I20260925 09:23:10.822093 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.821437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47440)
0925 09:23:10.821679 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.821783 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260925 09:23:10.823259 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.822808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47446)
0925 09:23:10.823050 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.823139 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260925 09:23:10.823467 17017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.855834 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 2
I20260925 09:23:10.857165 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.856546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47452)
0925 09:23:10.856815 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.856941 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260925 09:23:10.858459 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.858015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47458)
0925 09:23:10.858197 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.858300 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 09:23:10.858749 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
W20260925 09:23:10.960520 17025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:10.960842 17025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:10.961001 17025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 09:23:10.965065 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 3
I20260925 09:23:10.966338 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.965767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47470)
0925 09:23:10.966011 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.966140 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 09:23:10.966671 17017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 4
I20260925 09:23:10.967628 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.967175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47486)
0925 09:23:10.967361 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.967478 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 09:23:10.967854 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.967356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47494)
0925 09:23:10.967580 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.967692 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260925 09:23:10.967885 16943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
I20260925 09:23:10.969085 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:10.968650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47508)
0925 09:23:10.968812 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:10.968898 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:23:10.969298 17017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:46123: Network error: Client connection negotiation failed: client connection to 127.3.82.60:46123: connect: Connection refused (error 111)
W20260925 09:23:10.969924 17025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:10.970017 17025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:10.970104 17025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:10.970198 17025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:10.970286 17025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:10.982728 17025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42129,127.3.82.61:45823,127.3.82.60:46123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab
--principal=kudu/127.3.82.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:46123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.82.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:10.984449 17025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:10.986191 17025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:11.000139 17030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:11.000201 17031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:11.007004 17033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:11.007547 17025 server_base.cc:1034] running on GCE node
I20260925 09:23:11.008540 17025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:11.010362 17025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:11.011601 17025 hybrid_clock.cc:648] HybridClock initialized: now 1790328191011557 us; error 38 us; skew 500 ppm
I20260925 09:23:11.012104 17025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328191, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:11.015957 17025 init.cc:377] Logged in from keytab as kudu/127.3.82.60@KRBTEST.COM (short username kudu)
I20260925 09:23:11.017729 17025 webserver.cc:460] Webserver started at http://127.3.82.60:44349/ using document root <none> and password file <none>
I20260925 09:23:11.018527 17025 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:11.018679 17025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:11.019014 17025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:11.022255 17025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data/instance:
uuid: "82620c83e2ad47f4b5870d18c79bd55e"
format_stamp: "Formatted at 2026-09-25 09:23:11 on dist-test-slave-4wm5"
server_key: "dac5420e5e2b6a13299243f590194ba4"
server_key_iv: "f399652a610c2a3cc3d5c4b314bf0ea2"
server_key_version: "encryptionkey@0"
I20260925 09:23:11.023130 17025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal/instance:
uuid: "82620c83e2ad47f4b5870d18c79bd55e"
format_stamp: "Formatted at 2026-09-25 09:23:11 on dist-test-slave-4wm5"
server_key: "dac5420e5e2b6a13299243f590194ba4"
server_key_iv: "f399652a610c2a3cc3d5c4b314bf0ea2"
server_key_version: "encryptionkey@0"
I20260925 09:23:11.028857 17025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:23:11.032781 17040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:11.034653 17025 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:11.034929 17025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal
uuid: "82620c83e2ad47f4b5870d18c79bd55e"
format_stamp: "Formatted at 2026-09-25 09:23:11 on dist-test-slave-4wm5"
server_key: "dac5420e5e2b6a13299243f590194ba4"
server_key_iv: "f399652a610c2a3cc3d5c4b314bf0ea2"
server_key_version: "encryptionkey@0"
I20260925 09:23:11.035180 17025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:11.053141 17025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:11.058568 17025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:11.058898 17025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:11.079372 17025 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.60:46123
I20260925 09:23:11.079401 17091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.60:46123 every 8 connection(s)
I20260925 09:23:11.080899 17025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
I20260925 09:23:11.084746 17092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:11.091143 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17025
I20260925 09:23:11.091351 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790327973894689-3400-0/minicluster-data/master-2/wal/instance
I20260925 09:23:11.091965 3400 external_mini_cluster.cc:1428] Setting key f0ef68247401403903b869dfba33618e
I20260925 09:23:11.091773 17092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328191, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:23:11.116349 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.095087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.3.82.62:42129)
0925 09:23:11.095653 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.095659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.096289 (+ 630us) server_negotiation.cc:407] Connection header received
0925 09:23:11.097808 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.097816 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.097957 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.098066 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.100091 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.101471 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.103283 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.103705 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.103845 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.107843 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.107891 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.107900 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.107954 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.112044 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.113011 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.113019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.113023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.113198 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.113895 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.113902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.113906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.114413 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.114609 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.115602 (+ 993us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.115861 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":279,"threads_started":1}
I20260925 09:23:11.117870 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.094269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.3.82.60:46123)
0925 09:23:11.095382 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.095400 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.095455 (+ 55us) server_negotiation.cc:407] Connection header received
0925 09:23:11.097465 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.097493 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.097809 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.098078 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.100132 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.102091 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.103610 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.104237 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.104402 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.109212 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.109225 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.109249 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.109297 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.113252 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.114233 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.114258 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.114262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.114450 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.114932 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.114936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.114938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.115352 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.115728 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.116122 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.116390 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":572,"threads_started":1}
I20260925 09:23:11.117933 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.095120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:42129 (local address 127.0.0.1:44752)
0925 09:23:11.096115 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:11.096168 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:11.097495 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:11.098162 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:11.098169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:11.099153 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:11.099791 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.099824 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.101722 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.101729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.103065 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.103077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.103977 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.103984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.104187 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.104989 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:11.105022 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:11.107532 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:11.112241 (+ 4709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.112253 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.112263 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.112856 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.113397 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.113402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.113405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.113678 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.114607 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:11.114617 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:11.115167 (+ 550us) client_negotiation.cc:769] Sending connection context
0925 09:23:11.115477 (+ 310us) client_negotiation.cc:240] Negotiation successful
0925 09:23:11.116082 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":550,"threads_started":1}
I20260925 09:23:11.119876 17092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328191, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:11.136350 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.120809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.3.82.61:45823)
0925 09:23:11.121031 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.121039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.121063 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:23:11.121149 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.121155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.121246 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.121365 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.122435 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.123119 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.124704 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.125244 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.125349 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.129455 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.129461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.129464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.129499 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.133074 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.134147 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.134153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.134156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.134304 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.134854 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.134861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.134866 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.135233 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.135348 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.135882 (+ 534us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.136050 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:23:11.136548 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.120687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:45823 (local address 127.0.0.1:55986)
0925 09:23:11.120937 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:11.120953 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:11.121042 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:11.121389 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:11.121394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:11.121997 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:11.122228 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.122241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.123281 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.123290 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.124525 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.124539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.125417 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.125423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.125682 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.126423 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:11.126447 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:11.129222 (+ 2775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:11.133245 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.133252 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.133257 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.133996 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.134433 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.134439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.134445 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.134702 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.135371 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:11.135378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:11.135569 (+ 191us) client_negotiation.cc:769] Sending connection context
0925 09:23:11.135783 (+ 214us) client_negotiation.cc:240] Negotiation successful
0925 09:23:11.136117 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260925 09:23:11.137635 17092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:23:11.139941 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.121540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.3.82.62:42129)
0925 09:23:11.121708 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.121714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.121881 (+ 167us) server_negotiation.cc:407] Connection header received
0925 09:23:11.122020 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.122023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.122097 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.122226 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.123882 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.124711 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.126599 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.127023 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.127122 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.132775 (+ 5653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.132782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.132785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.132827 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.136592 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.137493 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.137500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.137506 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.137703 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.138190 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.138198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.138202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.138617 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.138917 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.139312 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.139578 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:23:11.145126 16943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 4
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328191, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:23:11.157572 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.138466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47518)
0925 09:23:11.138623 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:11.138642 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:11.138741 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:11.139028 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:11.139032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:11.139477 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:11.139604 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.139615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.140998 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.141004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.142741 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.142760 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.143914 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.143924 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.144191 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.144971 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:11.144994 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:11.149660 (+ 4666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:11.154274 (+ 4614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.154280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.154286 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.154807 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.155284 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.155290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.155294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.155479 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.156263 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:11.156270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:11.156440 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 09:23:11.156617 (+ 177us) client_negotiation.cc:240] Negotiation successful
0925 09:23:11.157019 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 09:23:11.158726 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.138556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.3.82.60:46123)
0925 09:23:11.138743 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.138749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.138764 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:23:11.138857 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.138861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.138919 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.139015 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.139841 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.140855 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.142934 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.143703 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.143859 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.149928 (+ 6069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.149937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.149947 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.149992 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.154078 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.154950 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.154957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.154962 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.155114 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.155611 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.155616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.155620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.156010 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.156183 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.157948 (+ 1765us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.158305 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:23:11.163905 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.145810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47530)
0925 09:23:11.146088 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:11.146135 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:11.146261 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:11.147981 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:11.147986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:11.148496 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:11.148734 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.148749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.150881 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.150888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.152554 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.152576 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.153429 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.153435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.153624 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.154265 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:11.154283 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:11.156647 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:11.160665 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.160671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.160675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.161060 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.161563 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.161570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.161574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.161761 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.162658 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:11.162664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:11.162871 (+ 207us) client_negotiation.cc:769] Sending connection context
0925 09:23:11.163042 (+ 171us) client_negotiation.cc:240] Negotiation successful
0925 09:23:11.163492 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 09:23:11.163969 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.146325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47530 (local address 127.3.82.60:46123)
0925 09:23:11.147597 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.147602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.147636 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:23:11.147708 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.147712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.147782 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.148958 (+ 1176us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.149252 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.150713 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.152767 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.153268 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.153406 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.156889 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.156894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.156897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.156924 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.160509 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.161191 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.161198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.161201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.161342 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.161981 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.161987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.161990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.162502 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.162664 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.163218 (+ 554us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.163473 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":888,"threads_started":1}
I20260925 09:23:11.165743 17092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: Bootstrap starting.
I20260925 09:23:11.168937 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.142032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.3.82.61:45823)
0925 09:23:11.146573 (+ 4541us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.146579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.146597 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:23:11.146655 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.146660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.146739 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.146843 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.150043 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.151938 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.154437 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.155140 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.155295 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.161566 (+ 6271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.161571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.161574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.161617 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.165403 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.166480 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.166487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.166490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.166665 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.167185 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.167191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.167195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.167611 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.167980 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.168283 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.168563 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4408}
I20260925 09:23:11.169740 16943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: Bootstrap starting.
I20260925 09:23:11.170496 17092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:11.171794 17092 log.cc:824] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:11.174609 16943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:11.174927 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:23:11.175349 17092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: No bootstrap required, opened a new log
I20260925 09:23:11.175979 16943 log.cc:824] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:11.179801 16943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: No bootstrap required, opened a new log
I20260925 09:23:11.187786 17092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.188468 17092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:11.188601 17092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82620c83e2ad47f4b5870d18c79bd55e, State: Initialized, Role: FOLLOWER
I20260925 09:23:11.189733 17092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.192179 16943 raft_consensus.cc:348] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.192628 16943 raft_consensus.cc:374] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:11.192763 16943 raft_consensus.cc:729] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b55936e1289548fc870e990cc7a23ddf, State: Initialized, Role: FOLLOWER
I20260925 09:23:11.193656 16943 consensus_queue.cc:260] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.196071 17107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.196698 17107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.197820 17092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:11.204469 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.182637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44774 (local address 127.3.82.62:42129)
0925 09:23:11.182895 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.182911 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.183450 (+ 539us) server_negotiation.cc:407] Connection header received
0925 09:23:11.183807 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.183813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.183903 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.183986 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.186527 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.187766 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.190067 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.190808 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.190915 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.196676 (+ 5761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.196687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.196695 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.196755 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.200888 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.201813 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.201839 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.201843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.202016 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.202500 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.202505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.202508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.202888 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.203162 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.203793 (+ 631us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.204045 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:23:11.206429 17109 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.206889 17109 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.209043 16943 sys_catalog.cc:564] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:11.218604 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.183620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.3.82.61:45823)
0925 09:23:11.183820 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.183826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.191081 (+ 7255us) server_negotiation.cc:407] Connection header received
0925 09:23:11.192121 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.192131 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.192259 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.192358 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.197978 (+ 5620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.199151 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.201383 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.202094 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.202206 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.208649 (+ 6443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.208661 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.208671 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.208738 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.213443 (+ 4705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.214705 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.214714 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.214721 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.214943 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.215539 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.215546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.215553 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.216072 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.217471 (+ 1399us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.217670 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.218072 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:23:11.222189 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.191217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.3.82.60:46123)
0925 09:23:11.191380 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.191385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.197486 (+ 6101us) server_negotiation.cc:407] Connection header received
0925 09:23:11.197808 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.197817 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.197986 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.198082 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.200501 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.202129 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.205879 (+ 3750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.207105 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.207321 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.211426 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.211437 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.211443 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.211505 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.216716 (+ 5211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.218055 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.218066 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.218073 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.218303 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.219398 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.219408 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.219414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.219967 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.220302 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.221214 (+ 912us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.221553 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260925 09:23:11.229054 16897 tablet.cc:2369] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:23:11.233199 17129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:23:11.233361 17129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:23:11.235391 16876 master_runner.cc:186] Error getting master registration for 127.3.82.61:45823: OK, instance_id { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" instance_seqno: 1790328190530534 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:23:11.248071 16876 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 09:23:11.250907 17131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:23:11.251192 17131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:23:11.236702 17025 master_runner.cc:186] Error getting master registration for 127.3.82.61:45823: OK, instance_id { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" instance_seqno: 1790328190530534 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:23:11.252444 17025 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:23:11.268517 17017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } attempt: 5
Sep 25 09:23:11 dist-test-slave-4wm5 krb5kdc[16863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328190, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:23:11.287140 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.269148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:46123 (local address 127.0.0.1:47534)
0925 09:23:11.269420 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:11.269441 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:11.269556 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:11.269961 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:11.269965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:11.270510 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:11.270739 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.270752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.272498 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.272505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.274059 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:11.274082 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.275154 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.275161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:11.275365 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.276160 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:11.276187 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:11.279260 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:11.283993 (+ 4733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.284001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.284006 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.284613 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.285112 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:11.285117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:11.285121 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:11.285298 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:11.286031 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:11.286037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:11.286207 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 09:23:11.286392 (+ 185us) client_negotiation.cc:240] Negotiation successful
0925 09:23:11.286709 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260925 09:23:11.287216 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:11.269316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.3.82.60:46123)
0925 09:23:11.269501 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:11.269507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:11.269527 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:23:11.269698 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:11.269704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:11.269783 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:11.269931 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:11.270915 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.272315 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.274247 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:11.274952 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:11.275102 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:11.279553 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:11.279562 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:11.279567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:11.279613 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:11.283785 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.284774 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.284782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.284787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.284962 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:11.285426 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:11.285435 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:11.285439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:11.285897 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:11.286112 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:11.286543 (+ 431us) server_negotiation.cc:299] Negotiation successful
0925 09:23:11.286784 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:23:11.293481 17017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e: Bootstrap starting.
I20260925 09:23:11.298096 17017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:11.299547 17017 log.cc:824] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:11.303323 17017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e: No bootstrap required, opened a new log
W20260925 09:23:11.311268 17046 tablet.cc:2369] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:11.315412 17017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.315937 17017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:11.316083 17017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d62f6bbfe3745eeb0b0ecc6c5b4829e, State: Initialized, Role: FOLLOWER
I20260925 09:23:11.316834 17017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.318112 17135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.318609 17135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.319574 17017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:11.327993 17135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:23:11.328312 17135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.330047 17135 leader_election.cc:288] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b55936e1289548fc870e990cc7a23ddf (127.3.82.62:42129), 82620c83e2ad47f4b5870d18c79bd55e (127.3.82.60:46123)
I20260925 09:23:11.331743 16918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b55936e1289548fc870e990cc7a23ddf" is_pre_election: true
I20260925 09:23:11.332422 16918 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d62f6bbfe3745eeb0b0ecc6c5b4829e in term 0.
I20260925 09:23:11.333173 17067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82620c83e2ad47f4b5870d18c79bd55e" is_pre_election: true
I20260925 09:23:11.334512 16969 leader_election.cc:302] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d62f6bbfe3745eeb0b0ecc6c5b4829e, b55936e1289548fc870e990cc7a23ddf; no voters:
I20260925 09:23:11.333804 17067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d62f6bbfe3745eeb0b0ecc6c5b4829e in term 0.
I20260925 09:23:11.340281 17135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:23:11.340493 17135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:23:11.340615 17135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:11.344488 17135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.345995 16918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b55936e1289548fc870e990cc7a23ddf"
I20260925 09:23:11.346248 16918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:11.349969 16918 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d62f6bbfe3745eeb0b0ecc6c5b4829e in term 1.
I20260925 09:23:11.350596 16969 leader_election.cc:302] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d62f6bbfe3745eeb0b0ecc6c5b4829e, b55936e1289548fc870e990cc7a23ddf; no voters:
I20260925 09:23:11.356114 17135 leader_election.cc:288] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [CANDIDATE]: Term 1 election: Requested vote from peers b55936e1289548fc870e990cc7a23ddf (127.3.82.62:42129), 82620c83e2ad47f4b5870d18c79bd55e (127.3.82.60:46123)
I20260925 09:23:11.357072 17135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:11.357599 17135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [term 1 LEADER]: Becoming Leader. State: Replica: 1d62f6bbfe3745eeb0b0ecc6c5b4829e, State: Running, Role: LEADER
I20260925 09:23:11.358414 17135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } }
I20260925 09:23:11.367254 17067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82620c83e2ad47f4b5870d18c79bd55e"
I20260925 09:23:11.367538 17067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:11.371354 17067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d62f6bbfe3745eeb0b0ecc6c5b4829e in term 1.
I20260925 09:23:11.376874 17138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d62f6bbfe3745eeb0b0ecc6c5b4829e. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.377485 17138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:11.384864 17143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:11.402628 17143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:11.424036 17151 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:23:11.440076 17067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [term 1 FOLLOWER]: Refusing update from remote peer 1d62f6bbfe3745eeb0b0ecc6c5b4829e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:11.441677 16918 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [term 1 FOLLOWER]: Refusing update from remote peer 1d62f6bbfe3745eeb0b0ecc6c5b4829e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:11.443034 17138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:11.443410 17135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [LEADER]: Connected to new peer: Peer: permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:11.471562 17107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d62f6bbfe3745eeb0b0ecc6c5b4829e. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.471895 17107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.473061 17109 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d62f6bbfe3745eeb0b0ecc6c5b4829e. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.473415 17109 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.476909 17107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.477210 17107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.477504 17150 mvcc.cc:205] Tried to move back new op lower bound from 7333184272087871488 to 7333184271819005952. Current Snapshot: MvccSnapshot[applied={T|T < 7333184272087871488}]
I20260925 09:23:11.482272 17138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.482605 17138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:11.482820 17138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.483086 17138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:11.484558 17143 catalog_manager.cc:1347] Generated new cluster ID: 61aebceb6d724272b2d9ab6e9827b592
I20260925 09:23:11.484685 17143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:23:11.496423 17153 mvcc.cc:205] Tried to move back new op lower bound from 7333184272087871488 to 7333184271819005952. Current Snapshot: MvccSnapshot[applied={T|T < 7333184272087871488}]
I20260925 09:23:11.510802 17143 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:23:11.513016 17143 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:23:11.516566 17109 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b55936e1289548fc870e990cc7a23ddf" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42129 } } peers { permanent_uuid: "1d62f6bbfe3745eeb0b0ecc6c5b4829e" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 45823 } } peers { permanent_uuid: "82620c83e2ad47f4b5870d18c79bd55e" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 46123 } } }
I20260925 09:23:11.516811 17109 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:23:11.544862 17143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d62f6bbfe3745eeb0b0ecc6c5b4829e: Generated new TSK 0
I20260925 09:23:11.546234 17143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:23:12.235381 17129 catalog_manager.cc:1259] Loaded cluster ID: 61aebceb6d724272b2d9ab6e9827b592
I20260925 09:23:12.235525 17129 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: loading cluster ID for follower catalog manager: success
I20260925 09:23:12.238179 17129 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: acquiring CA information for follower catalog manager: success
I20260925 09:23:12.240243 17129 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82620c83e2ad47f4b5870d18c79bd55e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:23:12.253464 17131 catalog_manager.cc:1259] Loaded cluster ID: 61aebceb6d724272b2d9ab6e9827b592
I20260925 09:23:12.253628 17131 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: loading cluster ID for follower catalog manager: success
I20260925 09:23:12.256489 17131 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: acquiring CA information for follower catalog manager: success
I20260925 09:23:12.258713 17131 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b55936e1289548fc870e990cc7a23ddf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:23:12.700551 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16876
I20260925 09:23:12.701903 16938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:12.814903 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16876
I20260925 09:23:12.837200 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 16950
I20260925 09:23:12.838764 17012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:12.951148 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 16950
I20260925 09:23:12.972559 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17025
I20260925 09:23:12.973737 17087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:13.089299 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17025
2026-09-25T09:23:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4977 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:23:13 dist-test-slave-4wm5 krb5kdc[17169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:23:13 dist-test-slave-4wm5 krb5kdc[17169](info): set up 2 sockets
Sep 25 09:23:13 dist-test-slave-4wm5 krb5kdc[17169](info): commencing operation
krb5kdc: starting...
W20260925 09:23:14.282593 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:23:14 dist-test-slave-4wm5 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:23:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:23:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:23:14.543071 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:40765
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:41239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:40765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:14.886845 17182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:14.887164 17182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:14.887324 17182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:14.894804 17182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:14.894922 17182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:14.895032 17182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:14.895114 17182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:14.895179 17182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:14.905396 17182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:41239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:40765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:14.906850 17182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:14.908140 17182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:14.920782 17187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:14.920897 17188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:14.921124 17190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:14.921644 17182 server_base.cc:1034] running on GCE node
I20260925 09:23:14.922555 17182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:14.924229 17182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:14.925469 17182 hybrid_clock.cc:648] HybridClock initialized: now 1790328194925410 us; error 44 us; skew 500 ppm
I20260925 09:23:14.925936 17182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:14 dist-test-slave-4wm5 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328194, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:14.933950 17182 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:14.935739 17182 webserver.cc:460] Webserver started at http://127.3.82.62:34305/ using document root <none> and password file <none>
I20260925 09:23:14.936542 17182 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:14.936700 17182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:14.937043 17182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:14.940321 17182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "c4d5df9dc3724146b5fa785e932d03d9"
format_stamp: "Formatted at 2026-09-25 09:23:14 on dist-test-slave-4wm5"
server_key: "38f152066b53383422bbe06c63fdf0d9"
server_key_iv: "3d7d04c2919fa2124fc33f37c7cc8e26"
server_key_version: "encryptionkey@0"
I20260925 09:23:14.941150 17182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "c4d5df9dc3724146b5fa785e932d03d9"
format_stamp: "Formatted at 2026-09-25 09:23:14 on dist-test-slave-4wm5"
server_key: "38f152066b53383422bbe06c63fdf0d9"
server_key_iv: "3d7d04c2919fa2124fc33f37c7cc8e26"
server_key_version: "encryptionkey@0"
I20260925 09:23:14.947358 17182 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260925 09:23:14.951750 17197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:14.953474 17182 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:23:14.953755 17182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "c4d5df9dc3724146b5fa785e932d03d9"
format_stamp: "Formatted at 2026-09-25 09:23:14 on dist-test-slave-4wm5"
server_key: "38f152066b53383422bbe06c63fdf0d9"
server_key_iv: "3d7d04c2919fa2124fc33f37c7cc8e26"
server_key_version: "encryptionkey@0"
I20260925 09:23:14.954118 17182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:14.978120 17182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:14.983543 17182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:14.983912 17182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:15.006901 17182 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:40765
I20260925 09:23:15.006914 17248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:40765 every 8 connection(s)
I20260925 09:23:15.008599 17182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:15.013026 17249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:15.018113 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17182
I20260925 09:23:15.018366 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:23:15.019033 3400 external_mini_cluster.cc:1428] Setting key 12db782c4179121e0891ca4649d7daf3
I20260925 09:23:15.025179 17249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9: Bootstrap starting.
I20260925 09:23:15.029251 17249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:15.030645 17249 log.cc:824] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:15.034785 17249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9: No bootstrap required, opened a new log
Sep 25 09:23:15 dist-test-slave-4wm5 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:15.043618 17249 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d5df9dc3724146b5fa785e932d03d9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40765 } }
I20260925 09:23:15.044042 17249 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:15.044175 17249 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d5df9dc3724146b5fa785e932d03d9, State: Initialized, Role: FOLLOWER
I20260925 09:23:15.044940 17249 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d5df9dc3724146b5fa785e932d03d9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40765 } }
I20260925 09:23:15.045241 17249 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:23:15.045384 17249 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:23:15.045598 17249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:15.046468 17252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:15.021371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.3.82.62:40765)
0925 09:23:15.022329 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:15.022342 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:15.022408 (+ 66us) server_negotiation.cc:407] Connection header received
0925 09:23:15.024216 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:15.024241 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:15.024695 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:15.025057 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:15.027172 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:15.028578 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:15.030638 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:15.031829 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:15.032074 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:15.037388 (+ 5314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:15.037399 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:15.037416 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:15.037496 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:15.041249 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:15.042228 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:15.042236 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:15.042245 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:15.042452 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:15.042892 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:15.042896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:15.042898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:15.043494 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:15.043826 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:15.044405 (+ 579us) server_negotiation.cc:299] Negotiation successful
0925 09:23:15.044901 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":456,"threads_started":1}
I20260925 09:23:15.049023 17249 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d5df9dc3724146b5fa785e932d03d9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40765 } }
I20260925 09:23:15.049674 17249 leader_election.cc:302] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4d5df9dc3724146b5fa785e932d03d9; no voters:
I20260925 09:23:15.050450 17249 leader_election.cc:288] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:23:15.050598 17254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:15.051494 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:23:15.051972 17254 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d5df9dc3724146b5fa785e932d03d9, State: Running, Role: LEADER
I20260925 09:23:15.052582 17254 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d5df9dc3724146b5fa785e932d03d9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40765 } }
I20260925 09:23:15.052919 17249 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:15.057101 17255 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4d5df9dc3724146b5fa785e932d03d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d5df9dc3724146b5fa785e932d03d9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40765 } } }
I20260925 09:23:15.057505 17255 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:15.058117 17256 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4d5df9dc3724146b5fa785e932d03d9. Latest consensus state: current_term: 1 leader_uuid: "c4d5df9dc3724146b5fa785e932d03d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d5df9dc3724146b5fa785e932d03d9" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40765 } } }
I20260925 09:23:15.058444 17256 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:15.060617 17263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:15.066114 17263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:15.076874 17263 catalog_manager.cc:1347] Generated new cluster ID: 254a90594be942c38cbae23596b97279
I20260925 09:23:15.077045 17263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:23:15.087277 17263 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:23:15.088711 17263 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:23:15.095608 17263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4d5df9dc3724146b5fa785e932d03d9: Generated new TSK 0
I20260925 09:23:15.096598 17263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:23:15.115242 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17182
I20260925 09:23:15.116540 17244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:15.215680 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17182
2026-09-25T09:23:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2120 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:23:15 dist-test-slave-4wm5 krb5kdc[17277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:23:15 dist-test-slave-4wm5 krb5kdc[17277](info): set up 2 sockets
Sep 25 09:23:15 dist-test-slave-4wm5 krb5kdc[17277](info): commencing operation
krb5kdc: starting...
W20260925 09:23:16.334533 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:23:16 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:23:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:23:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:23:16.594254 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37813
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:16.943874 17290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:16.944280 17290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:16.944444 17290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:16.952040 17290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:16.952212 17290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:16.952320 17290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:16.952414 17290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:16.952492 17290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:16.962970 17290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:16.964493 17290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:16.965899 17290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:16.979142 17295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:16.979192 17298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:16.979216 17296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:16.980829 17290 server_base.cc:1034] running on GCE node
I20260925 09:23:16.981797 17290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:16.983547 17290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:16.984836 17290 hybrid_clock.cc:648] HybridClock initialized: now 1790328196984791 us; error 42 us; skew 500 ppm
I20260925 09:23:16.985296 17290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:16 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328196, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:16.988883 17290 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:16.990828 17290 webserver.cc:460] Webserver started at http://127.3.82.62:45477/ using document root <none> and password file <none>
I20260925 09:23:16.991590 17290 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:16.991739 17290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:16.992057 17290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:16.995337 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "2833e06b4a584d7b9373387c85ed8056"
format_stamp: "Formatted at 2026-09-25 09:23:16 on dist-test-slave-4wm5"
server_key: "d6e8933a43654a738ee61e5a29360c1a"
server_key_iv: "b6e7eca6e53dedba9a708b8f4c41cdf1"
server_key_version: "encryptionkey@0"
I20260925 09:23:16.996192 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "2833e06b4a584d7b9373387c85ed8056"
format_stamp: "Formatted at 2026-09-25 09:23:16 on dist-test-slave-4wm5"
server_key: "d6e8933a43654a738ee61e5a29360c1a"
server_key_iv: "b6e7eca6e53dedba9a708b8f4c41cdf1"
server_key_version: "encryptionkey@0"
I20260925 09:23:17.001991 17290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 09:23:17.006752 17305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:17.008561 17290 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260925 09:23:17.008802 17290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2833e06b4a584d7b9373387c85ed8056"
format_stamp: "Formatted at 2026-09-25 09:23:16 on dist-test-slave-4wm5"
server_key: "d6e8933a43654a738ee61e5a29360c1a"
server_key_iv: "b6e7eca6e53dedba9a708b8f4c41cdf1"
server_key_version: "encryptionkey@0"
I20260925 09:23:17.009161 17290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:17.029173 17290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:17.034175 17290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:17.034485 17290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:17.054423 17290 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37813
I20260925 09:23:17.054446 17356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37813 every 8 connection(s)
I20260925 09:23:17.055871 17290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:17.060010 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17290
I20260925 09:23:17.060274 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:23:17.060395 17357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:17.061200 3400 external_mini_cluster.cc:1428] Setting key fcc2b910694f6059a4cc3470031c2630
I20260925 09:23:17.075309 17357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap starting.
Sep 25 09:23:17 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:17.080014 17357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:17.081643 17357 log.cc:824] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:17.086051 17357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: No bootstrap required, opened a new log
I20260925 09:23:17.087582 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:17.063504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.3.82.62:37813)
0925 09:23:17.064486 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:17.064498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:17.064542 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:23:17.066336 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:17.066360 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:17.066816 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:17.067264 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:17.068787 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:17.070220 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:17.071600 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:17.072366 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:17.072580 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:17.077227 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:17.077243 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:17.077269 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:17.077332 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:17.081872 (+ 4540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:17.082868 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:17.082876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:17.082883 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:17.083193 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:17.083633 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:17.083638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:17.083641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:17.084503 (+ 862us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:17.084837 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:17.085499 (+ 662us) server_negotiation.cc:299] Negotiation successful
0925 09:23:17.086028 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":433,"threads_started":1}
I20260925 09:23:17.096136 17357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:17.096570 17357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:17.096650 17357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2833e06b4a584d7b9373387c85ed8056, State: Initialized, Role: FOLLOWER
I20260925 09:23:17.097309 17357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:17.097536 17357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:23:17.097711 17357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:23:17.097965 17357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:17.100492 17357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:17.101091 17357 leader_election.cc:302] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2833e06b4a584d7b9373387c85ed8056; no voters:
I20260925 09:23:17.101768 17357 leader_election.cc:288] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:23:17.102049 17362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:17.103459 17362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 LEADER]: Becoming Leader. State: Replica: 2833e06b4a584d7b9373387c85ed8056, State: Running, Role: LEADER
I20260925 09:23:17.104117 17362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:17.104367 17357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:17.107482 17363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2833e06b4a584d7b9373387c85ed8056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } } }
I20260925 09:23:17.107532 17364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2833e06b4a584d7b9373387c85ed8056. Latest consensus state: current_term: 1 leader_uuid: "2833e06b4a584d7b9373387c85ed8056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } } }
I20260925 09:23:17.107975 17363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:17.107970 17364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:17.112433 17372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:17.117122 17372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:17.126374 17372 catalog_manager.cc:1347] Generated new cluster ID: eeeb1a5bfc8145eaa682b41ed5082793
I20260925 09:23:17.126557 17372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:23:17.137976 17372 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:23:17.139473 17372 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:23:17.155403 17372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Generated new TSK 0
I20260925 09:23:17.156407 17372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:23:17.223382 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37813
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:17.576108 17383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:17.576400 17383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:17.576505 17383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:17.584048 17383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:17.584169 17383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:17.584283 17383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:23:17.596930 17383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:17.598340 17383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:17.599613 17383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:17.613103 17388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:17.613201 17389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:17.613617 17391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:17.614332 17383 server_base.cc:1034] running on GCE node
I20260925 09:23:17.615056 17383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:17.616084 17383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:17.617288 17383 hybrid_clock.cc:648] HybridClock initialized: now 1790328197617236 us; error 59 us; skew 500 ppm
I20260925 09:23:17.617627 17383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:17 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:17.620934 17383 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:23:17.622643 17383 webserver.cc:460] Webserver started at http://127.3.82.1:36781/ using document root <none> and password file <none>
I20260925 09:23:17.623364 17383 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:17.623517 17383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:17.623831 17383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:17.627051 17383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "db587fbeda384badb57c60114def46e5"
format_stamp: "Formatted at 2026-09-25 09:23:17 on dist-test-slave-4wm5"
server_key: "c9d53d87660cf4a3a8e8316e82d7634e"
server_key_iv: "5fcbdd6dd85b7e0081edc2c37c8eb5e3"
server_key_version: "encryptionkey@0"
I20260925 09:23:17.627861 17383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "db587fbeda384badb57c60114def46e5"
format_stamp: "Formatted at 2026-09-25 09:23:17 on dist-test-slave-4wm5"
server_key: "c9d53d87660cf4a3a8e8316e82d7634e"
server_key_iv: "5fcbdd6dd85b7e0081edc2c37c8eb5e3"
server_key_version: "encryptionkey@0"
I20260925 09:23:17.633350 17383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 09:23:17.637821 17398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:17.639720 17383 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:17.639930 17383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "db587fbeda384badb57c60114def46e5"
format_stamp: "Formatted at 2026-09-25 09:23:17 on dist-test-slave-4wm5"
server_key: "c9d53d87660cf4a3a8e8316e82d7634e"
server_key_iv: "5fcbdd6dd85b7e0081edc2c37c8eb5e3"
server_key_version: "encryptionkey@0"
I20260925 09:23:17.640159 17383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:17.655053 17383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:17.659950 17383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:17.660259 17383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:17.661281 17383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:17.662894 17383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:17.663005 17383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:17.663156 17383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:17.663232 17383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:17.699199 17383 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:38121
I20260925 09:23:17.699294 17510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:38121 every 8 connection(s)
I20260925 09:23:17.700531 17383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:23:17.708571 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17383
I20260925 09:23:17.708796 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:23:17.709539 3400 external_mini_cluster.cc:1428] Setting key e3ff17ad4c26de8982c21b44a8fd4964
Sep 25 09:23:17 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:17.723177 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:17.703338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:42573 (local address 127.3.82.62:37813)
0925 09:23:17.703817 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:17.703822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:17.705097 (+ 1275us) server_negotiation.cc:407] Connection header received
0925 09:23:17.707004 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:17.707012 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:17.707116 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:17.707261 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:17.709213 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:17.710075 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:17.712210 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:17.712638 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:17.712784 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:17.716728 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:17.716733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:17.716737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:17.716775 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:17.719587 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:17.720546 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:17.720552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:17.720558 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:17.720691 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:17.721171 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:17.721177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:17.721180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:17.721505 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:17.721678 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:17.722498 (+ 820us) server_negotiation.cc:299] Negotiation successful
0925 09:23:17.722796 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":239,"threads_started":1}
I20260925 09:23:17.724226 17514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:17.703940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:42573)
0925 09:23:17.704801 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:17.704857 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:17.706671 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:17.707499 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:17.707507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:17.708123 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:17.709003 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:17.709033 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:17.710283 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:17.710287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:17.711994 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:17.712017 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:17.712889 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:17.712897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:17.713149 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:17.713976 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:17.714006 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:17.716440 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:17.719787 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:17.719799 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:17.719821 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:17.720386 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:17.720818 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:17.720823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:17.720827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:17.721049 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:17.721661 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:17.721671 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:17.722112 (+ 441us) client_negotiation.cc:769] Sending connection context
0925 09:23:17.722397 (+ 285us) client_negotiation.cc:240] Negotiation successful
0925 09:23:17.722987 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":375,"threads_started":1}
I20260925 09:23:17.726009 17511 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37813
I20260925 09:23:17.726354 17511 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:17.727262 17511 heartbeater.cc:511] Master 127.3.82.62:37813 requested a full tablet report, sending...
I20260925 09:23:17.730006 17322 ts_manager.cc:194] Registered new tserver with Master: db587fbeda384badb57c60114def46e5 (127.3.82.1:38121)
I20260925 09:23:17.732331 17322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:42573
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:23:17.769775 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37813
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:18.112742 17517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:18.113036 17517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:18.113204 17517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:18.120867 17517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:18.120999 17517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:18.121184 17517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:23:18.131299 17517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:18.132790 17517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:18.134209 17517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:18.148716 17522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:18.148942 17525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:18.149109 17523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:18.150261 17517 server_base.cc:1034] running on GCE node
I20260925 09:23:18.150985 17517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:18.152076 17517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:18.153353 17517 hybrid_clock.cc:648] HybridClock initialized: now 1790328198153311 us; error 43 us; skew 500 ppm
I20260925 09:23:18.153741 17517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:18 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:18.157510 17517 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:23:18.159533 17517 webserver.cc:460] Webserver started at http://127.3.82.2:37151/ using document root <none> and password file <none>
I20260925 09:23:18.160341 17517 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:18.160509 17517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:18.160859 17517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:18.164224 17517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "7c092ef305014c4dbd2176bc70917fe0"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "3d52e23de0ce588150f7b53631370f1e"
server_key_iv: "b6f1bbf105e62f65d8fc75b74fafdd69"
server_key_version: "encryptionkey@0"
I20260925 09:23:18.165124 17517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "7c092ef305014c4dbd2176bc70917fe0"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "3d52e23de0ce588150f7b53631370f1e"
server_key_iv: "b6f1bbf105e62f65d8fc75b74fafdd69"
server_key_version: "encryptionkey@0"
I20260925 09:23:18.171633 17517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260925 09:23:18.175920 17532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:18.177786 17517 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 09:23:18.178129 17517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "7c092ef305014c4dbd2176bc70917fe0"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "3d52e23de0ce588150f7b53631370f1e"
server_key_iv: "b6f1bbf105e62f65d8fc75b74fafdd69"
server_key_version: "encryptionkey@0"
I20260925 09:23:18.178378 17517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:18.195314 17517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:18.200292 17517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:18.200625 17517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:18.201694 17517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:18.203291 17517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:18.203419 17517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:18.203579 17517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:18.203655 17517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:18.240278 17517 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:39257
I20260925 09:23:18.240382 17644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:39257 every 8 connection(s)
I20260925 09:23:18.241658 17517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:23:18.245221 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17517
I20260925 09:23:18.245456 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:23:18.246053 3400 external_mini_cluster.cc:1428] Setting key 1778c817cae472ab7add9f1c1b1d2534
Sep 25 09:23:18 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:18.270289 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.244782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41713 (local address 127.3.82.62:37813)
0925 09:23:18.245385 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:18.245392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:18.247035 (+ 1643us) server_negotiation.cc:407] Connection header received
0925 09:23:18.249374 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:18.249379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:18.249461 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:18.249569 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:18.252271 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.253125 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.255963 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.256633 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.256776 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.261359 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:18.261369 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:18.261377 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:18.261431 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:18.265667 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:18.266800 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:18.266809 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:18.266816 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:18.267006 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:18.267593 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:18.267601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:18.267607 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:18.268128 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:18.268301 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:18.269418 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0925 09:23:18.269758 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":327,"threads_started":1}
I20260925 09:23:18.271207 17648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.245506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:41713)
0925 09:23:18.246714 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:18.246774 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:18.248991 (+ 2217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:18.249957 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:18.249973 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:18.250936 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:18.251901 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.251933 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.253348 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.253356 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.255759 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.255781 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.256925 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.256935 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.257319 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.258256 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:18.258281 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:18.260994 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:18.265979 (+ 4985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:18.265992 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:18.266009 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:18.266594 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:18.267184 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:18.267189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:18.267191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:18.267426 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:18.268338 (+ 912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:18.268346 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:18.268789 (+ 443us) client_negotiation.cc:769] Sending connection context
0925 09:23:18.269304 (+ 515us) client_negotiation.cc:240] Negotiation successful
0925 09:23:18.269919 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":358,"threads_started":1}
I20260925 09:23:18.273108 17645 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37813
I20260925 09:23:18.273452 17645 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:23:18.274360 17645 heartbeater.cc:511] Master 127.3.82.62:37813 requested a full tablet report, sending...
I20260925 09:23:18.276067 17322 ts_manager.cc:194] Registered new tserver with Master: 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
I20260925 09:23:18.277281 17322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:41713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:23:18.310179 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37813
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:18.658133 17651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:18.658591 17651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:18.658773 17651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:18.666620 17651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:18.666788 17651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:18.666975 17651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:23:18.677347 17651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:18.678772 17651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:18.680191 17651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:18.694685 17657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:18.694754 17659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:18.694669 17656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:18.695433 17651 server_base.cc:1034] running on GCE node
I20260925 09:23:18.696064 17651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:18.697026 17651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:18.698274 17651 hybrid_clock.cc:648] HybridClock initialized: now 1790328198698210 us; error 44 us; skew 500 ppm
I20260925 09:23:18.698690 17651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:18 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:18.702569 17651 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:23:18.704263 17651 webserver.cc:460] Webserver started at http://127.3.82.3:44805/ using document root <none> and password file <none>
I20260925 09:23:18.705003 17651 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:18.705150 17651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:18.705478 17651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:18.708626 17651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "478126b207d14c4a9ccf97201c4c96e8"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "345cb129ebf94f5b39d13feb83287648"
server_key_iv: "4393914df92673011e09f934d481eb6f"
server_key_version: "encryptionkey@0"
I20260925 09:23:18.709434 17651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "478126b207d14c4a9ccf97201c4c96e8"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "345cb129ebf94f5b39d13feb83287648"
server_key_iv: "4393914df92673011e09f934d481eb6f"
server_key_version: "encryptionkey@0"
I20260925 09:23:18.715075 17651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 09:23:18.718948 17666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:18.720625 17651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:23:18.720824 17651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "478126b207d14c4a9ccf97201c4c96e8"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "345cb129ebf94f5b39d13feb83287648"
server_key_iv: "4393914df92673011e09f934d481eb6f"
server_key_version: "encryptionkey@0"
I20260925 09:23:18.721069 17651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:18.736142 17511 heartbeater.cc:503] Master 127.3.82.62:37813 was elected leader, sending a full tablet report...
I20260925 09:23:18.739316 17651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:18.743682 17651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:18.743979 17651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:18.745061 17651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:18.746573 17651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:18.746649 17651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:18.746765 17651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:18.746809 17651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:18.783176 17651 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:44069
I20260925 09:23:18.783200 17778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:44069 every 8 connection(s)
I20260925 09:23:18.784601 17651 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:23:18.785724 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17651
I20260925 09:23:18.785974 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:23:18.786523 3400 external_mini_cluster.cc:1428] Setting key 1e769b03c1d3657113fb15c1a9025c62
Sep 25 09:23:18 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:18.808724 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.787533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:43521 (local address 127.3.82.62:37813)
0925 09:23:18.788348 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:18.788353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:18.789920 (+ 1567us) server_negotiation.cc:407] Connection header received
0925 09:23:18.791934 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:18.791940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:18.792031 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:18.792172 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:18.793927 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.794771 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.796510 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.796948 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.797101 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.801400 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:18.801407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:18.801411 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:18.801453 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:18.805163 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:18.806180 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:18.806184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:18.806187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:18.806318 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:18.806834 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:18.806839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:18.806846 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:18.807241 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:18.807393 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:18.808234 (+ 841us) server_negotiation.cc:299] Negotiation successful
0925 09:23:18.808426 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":521,"threads_started":1}
I20260925 09:23:18.809722 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.788459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:43521)
0925 09:23:18.789705 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:18.789752 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:18.791636 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:18.792452 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:18.792462 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:18.793075 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:18.793705 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.793724 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.794981 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.794985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.796321 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.796335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.797307 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.797317 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.797565 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.798458 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:18.798484 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:18.801046 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:18.805362 (+ 4316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:18.805371 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:18.805388 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:18.806054 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:18.806417 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:18.806421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:18.806423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:18.806624 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:18.807369 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:18.807376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:18.807750 (+ 374us) client_negotiation.cc:769] Sending connection context
0925 09:23:18.808057 (+ 307us) client_negotiation.cc:240] Negotiation successful
0925 09:23:18.808476 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"thread_start_us":670,"threads_started":1}
I20260925 09:23:18.811367 17779 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37813
I20260925 09:23:18.811692 17779 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:18.812461 17779 heartbeater.cc:511] Master 127.3.82.62:37813 requested a full tablet report, sending...
I20260925 09:23:18.814271 17322 ts_manager.cc:194] Registered new tserver with Master: 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069)
I20260925 09:23:18.815488 17322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:43521
I20260925 09:23:18.816383 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:23:18.835942 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.821047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.3.82.62:37813)
0925 09:23:18.821226 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:18.821231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:18.821677 (+ 446us) server_negotiation.cc:407] Connection header received
0925 09:23:18.821820 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:18.821980 (+ 160us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:18.822057 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:18.822151 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:18.823602 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.824320 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.826116 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.826588 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.826692 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.829728 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:18.829734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:18.829738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:18.829790 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:18.832804 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:18.833668 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:18.833672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:18.833675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:18.833816 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:18.834440 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:18.834444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:18.834446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:18.834830 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:18.835012 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:18.835477 (+ 465us) server_negotiation.cc:299] Negotiation successful
0925 09:23:18.835700 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:23:18.840951 17321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:23:18.848520 17321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:23:18.885769 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.862353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51670 (local address 127.3.82.1:38121)
0925 09:23:18.877769 (+ 15416us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:18.877777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:18.877802 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:23:18.877931 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:18.877942 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:18.878158 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:18.878358 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:18.879128 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.880657 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.883417 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.884499 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.884652 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.884915 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:18.885214 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 09:23:18.885398 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15253,"thread_start_us":15091,"threads_started":1}
I20260925 09:23:18.885907 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.862284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:39257 (local address 127.0.0.1:55422)
0925 09:23:18.875250 (+ 12966us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:18.875269 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:18.875381 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:18.878080 (+ 2699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:18.878084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:18.878115 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:18.878775 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.878794 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.880745 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.880751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.882784 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.882797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.884830 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.884840 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.885068 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.885104 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:23:18.885171 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 09:23:18.885399 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12849,"mutex_wait_us":80,"thread_start_us":11496,"threads_started":1}
I20260925 09:23:18.886220 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.875342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.3.82.3:44069)
0925 09:23:18.877042 (+ 1700us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:18.877069 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:18.877113 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:23:18.877258 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:18.877270 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:18.877599 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:18.877880 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:18.878779 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.880232 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.883236 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.884783 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.885365 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.885608 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:18.885676 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 09:23:18.885862 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1447,"thread_start_us":1261,"threads_started":1}
I20260925 09:23:18.886781 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.862254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:38121 (local address 127.0.0.1:51670)
0925 09:23:18.874380 (+ 12126us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:18.874438 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:18.874653 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:18.878375 (+ 3722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:18.878383 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:18.878414 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:18.878921 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.878933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.880804 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.880815 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.883260 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.883270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.884706 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.884712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.884947 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.885014 (+ 67us) client_negotiation.cc:769] Sending connection context
0925 09:23:18.885168 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 09:23:18.885405 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11723,"mutex_wait_us":85,"thread_start_us":13180,"threads_started":1}
I20260925 09:23:18.887238 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.875310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44069 (local address 127.0.0.1:40082)
0925 09:23:18.876139 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:18.876161 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:18.876295 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:18.877879 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:18.877887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:18.877916 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:18.878470 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.878492 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.880383 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.880387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.882991 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:18.883006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.885004 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.885010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:18.885159 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.885189 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:23:18.885238 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 09:23:18.886642 (+ 1404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":1275,"thread_start_us":456,"threads_started":1}
I20260925 09:23:18.887303 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:18.862859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55422 (local address 127.3.82.2:39257)
0925 09:23:18.877386 (+ 14527us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:18.877398 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:18.877432 (+ 34us) server_negotiation.cc:407] Connection header received
0925 09:23:18.877548 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:18.877558 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:18.877794 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:18.878666 (+ 872us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:18.878967 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.880568 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.882948 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:18.884580 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:18.886514 (+ 1934us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:18.886720 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:18.886803 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 09:23:18.886981 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14310,"thread_start_us":14170,"threads_started":1}
I20260925 09:23:18.890830 17446 tablet_service.cc:1467] Processing CreateTablet for tablet 099d55248e074e50a067584f971072bc (DEFAULT_TABLE table=test-table [id=c7c3e5c7f01b4402a1b66ffc1609922c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:18.890893 17714 tablet_service.cc:1467] Processing CreateTablet for tablet 099d55248e074e50a067584f971072bc (DEFAULT_TABLE table=test-table [id=c7c3e5c7f01b4402a1b66ffc1609922c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:18.892171 17446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099d55248e074e50a067584f971072bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:18.892338 17714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099d55248e074e50a067584f971072bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:18.892701 17580 tablet_service.cc:1467] Processing CreateTablet for tablet 099d55248e074e50a067584f971072bc (DEFAULT_TABLE table=test-table [id=c7c3e5c7f01b4402a1b66ffc1609922c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:18.894229 17580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099d55248e074e50a067584f971072bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:18.904551 17798 tablet_bootstrap.cc:492] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Bootstrap starting.
I20260925 09:23:18.907800 17799 tablet_bootstrap.cc:492] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Bootstrap starting.
I20260925 09:23:18.908128 17798 tablet_bootstrap.cc:654] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:18.909523 17798 log.cc:824] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:18.910172 17800 tablet_bootstrap.cc:492] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Bootstrap starting.
I20260925 09:23:18.911708 17799 tablet_bootstrap.cc:654] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:18.913112 17799 log.cc:824] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:18.913550 17800 tablet_bootstrap.cc:654] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:18.914127 17798 tablet_bootstrap.cc:492] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: No bootstrap required, opened a new log
I20260925 09:23:18.914635 17798 ts_tablet_manager.cc:1397] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260925 09:23:18.914991 17800 log.cc:824] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:18.918165 17799 tablet_bootstrap.cc:492] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: No bootstrap required, opened a new log
I20260925 09:23:18.918471 17800 tablet_bootstrap.cc:492] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: No bootstrap required, opened a new log
I20260925 09:23:18.918478 17799 ts_tablet_manager.cc:1397] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260925 09:23:18.918738 17800 ts_tablet_manager.cc:1397] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 09:23:18.926182 17799 raft_consensus.cc:348] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:18.926186 17798 raft_consensus.cc:348] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:18.926647 17798 raft_consensus.cc:374] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:18.926779 17798 raft_consensus.cc:729] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db587fbeda384badb57c60114def46e5, State: Initialized, Role: FOLLOWER
I20260925 09:23:18.926939 17799 raft_consensus.cc:374] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:18.927094 17799 raft_consensus.cc:729] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478126b207d14c4a9ccf97201c4c96e8, State: Initialized, Role: FOLLOWER
I20260925 09:23:18.927484 17798 consensus_queue.cc:260] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:18.927745 17799 consensus_queue.cc:260] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:18.929585 17779 heartbeater.cc:503] Master 127.3.82.62:37813 was elected leader, sending a full tablet report...
I20260925 09:23:18.930383 17799 ts_tablet_manager.cc:1428] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260925 09:23:18.930207 17800 raft_consensus.cc:348] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:18.930814 17800 raft_consensus.cc:374] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:18.930945 17800 raft_consensus.cc:729] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c092ef305014c4dbd2176bc70917fe0, State: Initialized, Role: FOLLOWER
I20260925 09:23:18.931895 17800 consensus_queue.cc:260] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:18.933917 17645 heartbeater.cc:503] Master 127.3.82.62:37813 was elected leader, sending a full tablet report...
I20260925 09:23:18.934300 17800 ts_tablet_manager.cc:1428] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260925 09:23:18.936070 17798 ts_tablet_manager.cc:1428] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
W20260925 09:23:18.952955 17512 tablet.cc:2369] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:23:18.994213 17646 tablet.cc:2369] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:23:19.037060 17780 tablet.cc:2369] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:19.085547 17804 raft_consensus.cc:482] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:23:19.085891 17804 raft_consensus.cc:504] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:19.087558 17804 leader_election.cc:288] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257), 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069)
I20260925 09:23:19.094648 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.088163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:58925 (local address 127.3.82.2:39257)
0925 09:23:19.088359 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.088363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.088998 (+ 635us) server_negotiation.cc:407] Connection header received
0925 09:23:19.089351 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.089361 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.089445 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.089515 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:19.090237 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.091032 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.092800 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.093590 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.093750 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.093881 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.094193 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.094353 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:23:19.094942 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.088072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:39257 (local address 127.3.82.1:58925)
0925 09:23:19.088843 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:19.088869 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:19.089033 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:19.089709 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:19.089712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:19.089734 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:19.090081 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.090089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.091172 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.091180 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.092639 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.092651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.093820 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.093867 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.094019 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.094043 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 09:23:19.094163 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 09:23:19.094572 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":438,"threads_started":1}
I20260925 09:23:19.095311 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.089336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:58477 (local address 127.3.82.3:44069)
0925 09:23:19.089555 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.089562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.089772 (+ 210us) server_negotiation.cc:407] Connection header received
0925 09:23:19.090036 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.090042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.090121 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.090216 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:19.090977 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.091715 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.093381 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.094248 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.094409 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.094513 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.094906 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.095058 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:23:19.095095 17806 raft_consensus.cc:482] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:23:19.095616 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.089050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44069 (local address 127.3.82.1:58477)
0925 09:23:19.089661 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:19.089676 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:19.089886 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:19.090383 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:19.090388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:19.090416 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:19.090815 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.090825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.091857 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.091863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.093225 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.093236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.094463 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.094470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.094647 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.094671 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 09:23:19.094783 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:23:19.095267 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":286,"threads_started":1}
I20260925 09:23:19.095639 17806 raft_consensus.cc:504] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:19.096299 17600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d55248e074e50a067584f971072bc" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c092ef305014c4dbd2176bc70917fe0" is_pre_election: true
I20260925 09:23:19.096714 17600 raft_consensus.cc:2454] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 0.
I20260925 09:23:19.097522 17401 leader_election.cc:302] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c092ef305014c4dbd2176bc70917fe0, db587fbeda384badb57c60114def46e5; no voters:
I20260925 09:23:19.098059 17804 raft_consensus.cc:2781] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:23:19.098222 17804 raft_consensus.cc:482] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:23:19.098323 17804 raft_consensus.cc:3037] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:19.098140 17734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d55248e074e50a067584f971072bc" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478126b207d14c4a9ccf97201c4c96e8" is_pre_election: true
I20260925 09:23:19.098687 17734 raft_consensus.cc:2454] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 0.
I20260925 09:23:19.100140 17806 leader_election.cc:288] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db587fbeda384badb57c60114def46e5 (127.3.82.1:38121), 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069)
I20260925 09:23:19.101733 17804 raft_consensus.cc:504] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:19.103046 17600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d55248e074e50a067584f971072bc" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c092ef305014c4dbd2176bc70917fe0"
I20260925 09:23:19.103323 17600 raft_consensus.cc:3037] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:19.103956 17804 leader_election.cc:288] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 election: Requested vote from peers 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257), 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069)
I20260925 09:23:19.104772 17734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d55248e074e50a067584f971072bc" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478126b207d14c4a9ccf97201c4c96e8"
I20260925 09:23:19.105022 17734 raft_consensus.cc:3037] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:19.107107 17600 raft_consensus.cc:2454] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 1.
I20260925 09:23:19.107625 17401 leader_election.cc:302] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c092ef305014c4dbd2176bc70917fe0, db587fbeda384badb57c60114def46e5; no voters:
I20260925 09:23:19.108392 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.098348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:45887 (local address 127.3.82.1:38121)
0925 09:23:19.098553 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.098559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.100221 (+ 1662us) server_negotiation.cc:407] Connection header received
0925 09:23:19.100492 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.100497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.100596 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.100666 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:19.101527 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.102718 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.105653 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.106942 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.107060 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.107182 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.107932 (+ 750us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.108098 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 09:23:19.108688 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.098053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:55019 (local address 127.3.82.3:44069)
0925 09:23:19.098651 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.098659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.100885 (+ 2226us) server_negotiation.cc:407] Connection header received
0925 09:23:19.101105 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.101114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.101205 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.101309 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:19.101959 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.103002 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.105924 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.107343 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.107482 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.107609 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.108242 (+ 633us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.108380 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444}
I20260925 09:23:19.108850 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.097874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:38121 (local address 127.3.82.2:45887)
0925 09:23:19.100121 (+ 2247us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:19.100142 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:19.100331 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:19.100857 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:19.100862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:19.100889 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:19.101313 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.101325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.102881 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.102886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.105434 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.105453 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.107411 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.107418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.107678 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.107739 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 09:23:19.108230 (+ 491us) client_negotiation.cc:240] Negotiation successful
0925 09:23:19.108384 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2096,"mutex_wait_us":5,"thread_start_us":2633,"threads_started":1}
I20260925 09:23:19.108736 17734 raft_consensus.cc:2454] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 1.
I20260925 09:23:19.108201 17804 raft_consensus.cc:2781] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:19.109349 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.098262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44069 (local address 127.3.82.2:55019)
0925 09:23:19.100774 (+ 2512us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:19.100789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:19.100945 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:19.101335 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:19.101339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:19.101363 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:19.101754 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.101770 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.103184 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.103189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.105676 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:19.105692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.107872 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.107878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:19.108047 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.108081 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 09:23:19.108859 (+ 778us) client_negotiation.cc:240] Negotiation successful
0925 09:23:19.109012 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2409,"thread_start_us":1490,"threads_started":1}
I20260925 09:23:19.109895 17804 raft_consensus.cc:686] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 1 LEADER]: Becoming Leader. State: Replica: db587fbeda384badb57c60114def46e5, State: Running, Role: LEADER
I20260925 09:23:19.110088 17734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d55248e074e50a067584f971072bc" candidate_uuid: "7c092ef305014c4dbd2176bc70917fe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478126b207d14c4a9ccf97201c4c96e8" is_pre_election: true
I20260925 09:23:19.110054 17466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099d55248e074e50a067584f971072bc" candidate_uuid: "7c092ef305014c4dbd2176bc70917fe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db587fbeda384badb57c60114def46e5" is_pre_election: true
I20260925 09:23:19.110428 17734 raft_consensus.cc:2379] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c092ef305014c4dbd2176bc70917fe0 in current term 1: Already voted for candidate db587fbeda384badb57c60114def46e5 in this term.
I20260925 09:23:19.110961 17804 consensus_queue.cc:237] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } }
I20260925 09:23:19.111385 17536 leader_election.cc:302] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c092ef305014c4dbd2176bc70917fe0; no voters: 478126b207d14c4a9ccf97201c4c96e8, db587fbeda384badb57c60114def46e5
I20260925 09:23:19.111778 17806 raft_consensus.cc:2726] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:23:19.116407 17322 catalog_manager.cc:5495] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to db587fbeda384badb57c60114def46e5 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "db587fbeda384badb57c60114def46e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } health_report { overall_health: UNKNOWN } } }
I20260925 09:23:19.174443 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.168786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.3.82.1:38121)
0925 09:23:19.168982 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.168987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.169007 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:23:19.169111 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.169115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.169197 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.169334 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:23:19.169721 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.170569 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.172165 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.172621 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.172725 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.173196 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:23:19.173743 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:23:19.173942 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.174050 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.174127 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:23:19.183182 17734 raft_consensus.cc:1261] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Refusing update from remote peer db587fbeda384badb57c60114def46e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:19.183234 17600 raft_consensus.cc:1261] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Refusing update from remote peer db587fbeda384badb57c60114def46e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:19.184223 17811 consensus_queue.cc:1035] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:19.184676 17804 consensus_queue.cc:1035] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:19.214233 17320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:19.214618 17320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:19.218700 17320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Sending DeleteTablet for 3 replicas of tablet 099d55248e074e50a067584f971072bc
I20260925 09:23:19.220207 17714 tablet_service.cc:1514] Processing DeleteTablet for tablet 099d55248e074e50a067584f971072bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:19 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:40082
I20260925 09:23:19.220258 17446 tablet_service.cc:1514] Processing DeleteTablet for tablet 099d55248e074e50a067584f971072bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:19 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:51670
I20260925 09:23:19.220945 17580 tablet_service.cc:1514] Processing DeleteTablet for tablet 099d55248e074e50a067584f971072bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:19 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:55422
I20260925 09:23:19.221113 17824 tablet_replica.cc:330] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: stopping tablet replica
I20260925 09:23:19.221199 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17383
I20260925 09:23:19.221604 17824 raft_consensus.cc:2229] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:19.222309 17824 raft_consensus.cc:2258] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:19.225646 17824 ts_tablet_manager.cc:1905] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:19.231551 17826 tablet_replica.cc:330] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: stopping tablet replica
I20260925 09:23:19.232268 17826 raft_consensus.cc:2229] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:19.233013 17825 tablet_replica.cc:330] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: stopping tablet replica
I20260925 09:23:19.233589 17826 raft_consensus.cc:2258] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:19.234133 17825 raft_consensus.cc:2229] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:23:19.236503 17824 ts_tablet_manager.cc:1918] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:19.236742 17824 log.cc:1196] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/099d55248e074e50a067584f971072bc
I20260925 09:23:19.237385 17824 ts_tablet_manager.cc:1939] T 099d55248e074e50a067584f971072bc P 478126b207d14c4a9ccf97201c4c96e8: Deleting consensus metadata
I20260925 09:23:19.237445 17825 raft_consensus.cc:2258] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:19.239671 17308 catalog_manager.cc:4841] TS 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069): tablet 099d55248e074e50a067584f971072bc (table test-table [id=c7c3e5c7f01b4402a1b66ffc1609922c]) successfully deleted
I20260925 09:23:19.239933 17825 ts_tablet_manager.cc:1905] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:19.236563 17826 ts_tablet_manager.cc:1905] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:19.251174 17826 ts_tablet_manager.cc:1918] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:19.251384 17826 log.cc:1196] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/099d55248e074e50a067584f971072bc
I20260925 09:23:19.251969 17826 ts_tablet_manager.cc:1939] T 099d55248e074e50a067584f971072bc P 7c092ef305014c4dbd2176bc70917fe0: Deleting consensus metadata
Sep 25 09:23:19 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:23:19.254040 17308 catalog_manager.cc:4841] TS 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257): tablet 099d55248e074e50a067584f971072bc (table test-table [id=c7c3e5c7f01b4402a1b66ffc1609922c]) successfully deleted
I20260925 09:23:19.259841 17825 ts_tablet_manager.cc:1918] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:19.260066 17825 log.cc:1196] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/099d55248e074e50a067584f971072bc
I20260925 09:23:19.260660 17825 ts_tablet_manager.cc:1939] T 099d55248e074e50a067584f971072bc P db587fbeda384badb57c60114def46e5: Deleting consensus metadata
I20260925 09:23:19.262784 17309 catalog_manager.cc:4841] TS db587fbeda384badb57c60114def46e5 (127.3.82.1:38121): tablet 099d55248e074e50a067584f971072bc (table test-table [id=c7c3e5c7f01b4402a1b66ffc1609922c]) successfully deleted
I20260925 09:23:19.264871 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.222536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51690 (local address 127.3.82.1:38121)
0925 09:23:19.222735 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.222741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.237554 (+ 14813us) server_negotiation.cc:407] Connection header received
0925 09:23:19.238170 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.238176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.238273 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.238358 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:19.243534 (+ 5176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.244826 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.247316 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.248114 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.248243 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.255209 (+ 6966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:19.255237 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:19.255264 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:19.255327 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:19.260393 (+ 5066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:19.261601 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:19.261616 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:19.261620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:19.261855 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:19.262405 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:19.262412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:19.262416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:19.263035 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:19.263393 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.264187 (+ 794us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.264487 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 09:23:19.265790 17506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:19.373562 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17383
I20260925 09:23:19.393960 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17517
Sep 25 09:23:19 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:23:19.410838 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.394817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.3.82.2:39257)
0925 09:23:19.395120 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.395127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.395147 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:23:19.395337 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.395347 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.395424 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.395536 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:19.396834 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.397602 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.399121 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.399553 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.399699 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.404301 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:19.404316 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:19.404332 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:19.404370 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:19.407804 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:19.408653 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:19.408660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:19.408662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:19.408789 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:19.409228 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:19.409252 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:19.409254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:19.409651 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:19.409952 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.410386 (+ 434us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.410570 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:23:19.411475 17640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:19.506399 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17517
I20260925 09:23:19.524538 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17651
Sep 25 09:23:19 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:23:19.542063 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:19.525526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.3.82.3:44069)
0925 09:23:19.525734 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:19.525740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:19.525760 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:23:19.526116 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:19.526120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:19.526187 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:19.526303 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:19.527815 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.528510 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.529854 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:19.530321 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:19.530473 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:19.535390 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:19.535410 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:19.535427 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:19.535477 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:19.538726 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:19.539658 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:19.539666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:19.539669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:19.539823 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:19.540270 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:19.540276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:19.540280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:19.540687 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:19.541024 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:19.541407 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 09:23:19.541651 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:23:19.542807 17774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:19.637684 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17651
I20260925 09:23:19.654976 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17290
I20260925 09:23:19.656342 17352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:19.772701 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17290
I20260925 09:23:19.794751 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37813
--webserver_interface=127.3.82.62
--webserver_port=45477
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:20.151055 17832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:20.151418 17832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:20.151583 17832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:20.159099 17832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:20.159252 17832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:20.159360 17832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:20.159459 17832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:20.159546 17832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:20.170629 17832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=45477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:20.172112 17832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:20.173530 17832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:20.202441 17838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:20.202455 17839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:20.202477 17841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:20.203557 17832 server_base.cc:1034] running on GCE node
I20260925 09:23:20.204579 17832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:20.206328 17832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:20.207583 17832 hybrid_clock.cc:648] HybridClock initialized: now 1790328200207509 us; error 61 us; skew 500 ppm
I20260925 09:23:20.208011 17832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:20 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328200, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:20.216239 17832 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:20.218495 17832 webserver.cc:460] Webserver started at http://127.3.82.62:45477/ using document root <none> and password file <none>
I20260925 09:23:20.219405 17832 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:20.219589 17832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:20.226475 17832 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:20.230234 17848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:20.231901 17832 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:23:20.232157 17832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2833e06b4a584d7b9373387c85ed8056"
format_stamp: "Formatted at 2026-09-25 09:23:16 on dist-test-slave-4wm5"
server_key: "d6e8933a43654a738ee61e5a29360c1a"
server_key_iv: "b6e7eca6e53dedba9a708b8f4c41cdf1"
server_key_version: "encryptionkey@0"
I20260925 09:23:20.233435 17832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:20.256839 17832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:20.262722 17832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:20.263168 17832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:20.286064 17832 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37813
I20260925 09:23:20.286162 17899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37813 every 8 connection(s)
I20260925 09:23:20.287855 17832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:20.291644 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17832
I20260925 09:23:20.292682 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:38121
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:37813
--webserver_port=36781
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:20.298199 17900 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:23:20.303560 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap starting.
I20260925 09:23:20.326514 17900 log.cc:824] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:20.340960 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:23:20.341621 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap complete.
I20260925 09:23:20.352326 17900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:20.353343 17900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2833e06b4a584d7b9373387c85ed8056, State: Initialized, Role: FOLLOWER
I20260925 09:23:20.354184 17900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:20.354477 17900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:23:20.354631 17900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:23:20.354849 17900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:23:20.359426 17900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:20.360057 17900 leader_election.cc:302] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2833e06b4a584d7b9373387c85ed8056; no voters:
I20260925 09:23:20.360780 17900 leader_election.cc:288] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:23:20.360934 17904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:23:20.362288 17904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 LEADER]: Becoming Leader. State: Replica: 2833e06b4a584d7b9373387c85ed8056, State: Running, Role: LEADER
I20260925 09:23:20.362908 17904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:20.363441 17900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:20.366007 17905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2833e06b4a584d7b9373387c85ed8056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } } }
I20260925 09:23:20.366055 17906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2833e06b4a584d7b9373387c85ed8056. Latest consensus state: current_term: 2 leader_uuid: "2833e06b4a584d7b9373387c85ed8056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } } }
I20260925 09:23:20.366477 17905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:20.366509 17906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:20.368161 17910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:20.377715 17910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:20.379225 17910 catalog_manager.cc:1259] Loaded cluster ID: eeeb1a5bfc8145eaa682b41ed5082793
I20260925 09:23:20.379335 17910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:23:20.382496 17910 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 09:23:20.400650 17910 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 09:23:20.425433 17910 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:23:20.426817 17921 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:23:20.427906 17910 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Loaded TSK: 0
I20260925 09:23:20.429186 17910 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:23:20.698486 17902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:20.698788 17902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:20.698941 17902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:20.706897 17902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:20.707015 17902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:20.707191 17902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:23:20.717684 17902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:38121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=36781
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:20.719285 17902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:20.720750 17902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:20.737859 17928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:20.737993 17927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:20.739084 17930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:20.739333 17902 server_base.cc:1034] running on GCE node
I20260925 09:23:20.740296 17902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:20.741809 17902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:20.743142 17902 hybrid_clock.cc:648] HybridClock initialized: now 1790328200743056 us; error 71 us; skew 500 ppm
I20260925 09:23:20.743631 17902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:20 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328200, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:20.748653 17902 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:23:20.750959 17902 webserver.cc:460] Webserver started at http://127.3.82.1:36781/ using document root <none> and password file <none>
I20260925 09:23:20.752040 17902 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:20.752218 17902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:20.760744 17902 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:23:20.765452 17937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:20.767674 17902 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:20.767946 17902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "db587fbeda384badb57c60114def46e5"
format_stamp: "Formatted at 2026-09-25 09:23:17 on dist-test-slave-4wm5"
server_key: "c9d53d87660cf4a3a8e8316e82d7634e"
server_key_iv: "5fcbdd6dd85b7e0081edc2c37c8eb5e3"
server_key_version: "encryptionkey@0"
I20260925 09:23:20.769630 17902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:20.791014 17902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:20.797729 17902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:20.798130 17902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:20.799492 17902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:20.801506 17902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:20.801652 17902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:20.801783 17902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:20.801889 17902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260925 09:23:20.841696 17902 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:38121
I20260925 09:23:20.841732 18049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:38121 every 8 connection(s)
I20260925 09:23:20.843801 17902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:23:20.844245 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 17902
I20260925 09:23:20.845112 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:39257
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:37813
--webserver_port=37151
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:23:20 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328200, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:20.882266 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:20.847975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:42581)
0925 09:23:20.858926 (+ 10951us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:20.858982 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:20.860983 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:20.863409 (+ 2426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:20.863424 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:20.864212 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:20.865028 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:20.865054 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:20.866877 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:20.866882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:20.868211 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:20.868225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:20.869373 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:20.869380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:20.869692 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:20.870738 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:20.870764 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:20.873466 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:20.877458 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:20.877468 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:20.877485 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:20.878021 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:20.878509 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:20.878516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:20.878524 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:20.878715 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:20.879539 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:20.879548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:20.879922 (+ 374us) client_negotiation.cc:769] Sending connection context
0925 09:23:20.880209 (+ 287us) client_negotiation.cc:240] Negotiation successful
0925 09:23:20.880830 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10648,"thread_start_us":10405,"threads_started":1}
I20260925 09:23:20.882270 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:20.849022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:42581 (local address 127.3.82.62:37813)
0925 09:23:20.858468 (+ 9446us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:20.858490 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:20.859162 (+ 672us) server_negotiation.cc:407] Connection header received
0925 09:23:20.861689 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:20.861705 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:20.862714 (+ 1009us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:20.863204 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:20.865282 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:20.866664 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:20.868408 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:20.869101 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:20.869329 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:20.873815 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:20.873875 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:20.873900 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:20.873963 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:20.877276 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:20.878194 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:20.878201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:20.878207 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:20.878379 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:20.878871 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:20.878875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:20.878877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:20.879386 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:20.879597 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:20.880486 (+ 889us) server_negotiation.cc:299] Negotiation successful
0925 09:23:20.880917 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9174,"thread_start_us":9022,"threads_started":1}
I20260925 09:23:20.885530 18050 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37813
I20260925 09:23:20.885911 18050 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:20.886813 18050 heartbeater.cc:511] Master 127.3.82.62:37813 requested a full tablet report, sending...
I20260925 09:23:20.889607 17865 ts_manager.cc:194] Registered new tserver with Master: db587fbeda384badb57c60114def46e5 (127.3.82.1:38121)
I20260925 09:23:20.891666 17865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:42581
W20260925 09:23:21.209720 18054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:21.210041 18054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:21.210137 18054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:21.217643 18054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:21.217763 18054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:21.217944 18054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:23:21.228279 18054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:39257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=37151
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:21.229589 18054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:21.230933 18054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:21.245347 18062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:21.245278 18059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:21.245314 18060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:21.246117 18054 server_base.cc:1034] running on GCE node
I20260925 09:23:21.246742 18054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:21.247774 18054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:21.249009 18054 hybrid_clock.cc:648] HybridClock initialized: now 1790328201248959 us; error 118 us; skew 500 ppm
I20260925 09:23:21.249397 18054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:21 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328201, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:21.252954 18054 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:23:21.261502 18054 webserver.cc:460] Webserver started at http://127.3.82.2:37151/ using document root <none> and password file <none>
I20260925 09:23:21.262777 18054 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:21.262969 18054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:21.269412 18054 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260925 09:23:21.273104 18069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:21.275177 18054 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260925 09:23:21.275439 18054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "7c092ef305014c4dbd2176bc70917fe0"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "3d52e23de0ce588150f7b53631370f1e"
server_key_iv: "b6f1bbf105e62f65d8fc75b74fafdd69"
server_key_version: "encryptionkey@0"
I20260925 09:23:21.276659 18054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:21.305804 18054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:21.310624 18054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:21.310925 18054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:21.312112 18054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:21.313555 18054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:21.313691 18054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:21.313817 18054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:21.313953 18054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260925 09:23:21.350461 18054 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:39257
I20260925 09:23:21.350550 18181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:39257 every 8 connection(s)
I20260925 09:23:21.352067 18054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:23:21.353564 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18054
I20260925 09:23:21.354362 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:44069
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:37813
--webserver_port=44805
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:23:21 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328201, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:21.376034 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.355264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:55923 (local address 127.3.82.62:37813)
0925 09:23:21.355459 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.355464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.356943 (+ 1479us) server_negotiation.cc:407] Connection header received
0925 09:23:21.359138 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.359145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.359277 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.359398 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:21.361049 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.361774 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.363663 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.364148 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.364397 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.369274 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:21.369281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:21.369285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:21.369336 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:21.372452 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:21.373445 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:21.373454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:21.373458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:21.373618 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:21.374159 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:21.374169 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:21.374171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:21.374462 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:21.374637 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.375414 (+ 777us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.375677 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:23:21.377305 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.355912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:55923)
0925 09:23:21.356758 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:21.356805 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:21.358775 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:21.359665 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:21.359674 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:21.360233 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:21.360848 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.360867 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.361987 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.361996 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.363471 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.363486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.364604 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.364614 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.365034 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.365921 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:21.365957 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:21.368961 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:21.372686 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:21.372695 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:21.372712 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:21.373212 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:21.373743 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:21.373748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:21.373753 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:21.374020 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:21.374605 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:21.374614 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:21.375015 (+ 401us) client_negotiation.cc:769] Sending connection context
0925 09:23:21.375308 (+ 293us) client_negotiation.cc:240] Negotiation successful
0925 09:23:21.375898 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":349,"threads_started":1}
I20260925 09:23:21.379030 18182 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37813
I20260925 09:23:21.379455 18182 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:21.380254 18182 heartbeater.cc:511] Master 127.3.82.62:37813 requested a full tablet report, sending...
I20260925 09:23:21.381770 17865 ts_manager.cc:194] Registered new tserver with Master: 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257)
I20260925 09:23:21.382732 17865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:55923
W20260925 09:23:21.718955 18185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:21.719424 18185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:21.719614 18185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:21.727576 18185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:21.727746 18185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:21.728029 18185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:23:21.739379 18185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:44069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=44805
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:21.740859 18185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:21.742542 18185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:21.757331 18190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:21.757279 18193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:21.757277 18191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:21.758240 18185 server_base.cc:1034] running on GCE node
I20260925 09:23:21.758915 18185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:21.759999 18185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:21.761229 18185 hybrid_clock.cc:648] HybridClock initialized: now 1790328201761142 us; error 81 us; skew 500 ppm
I20260925 09:23:21.761675 18185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:21 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328201, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:21.765728 18185 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:23:21.767867 18185 webserver.cc:460] Webserver started at http://127.3.82.3:44805/ using document root <none> and password file <none>
I20260925 09:23:21.768642 18185 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:21.768780 18185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:21.775703 18185 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:23:21.779192 18200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:21.780941 18185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:23:21.781105 18185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "478126b207d14c4a9ccf97201c4c96e8"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "345cb129ebf94f5b39d13feb83287648"
server_key_iv: "4393914df92673011e09f934d481eb6f"
server_key_version: "encryptionkey@0"
I20260925 09:23:21.782284 18185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:21.806622 18185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:21.811911 18185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:21.812287 18185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:21.813455 18185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:21.815146 18185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:21.815255 18185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:21.815414 18185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:21.815496 18185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:21.854467 18185 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:44069
I20260925 09:23:21.854614 18312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:44069 every 8 connection(s)
I20260925 09:23:21.856031 18185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:23:21.864547 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18185
Sep 25 09:23:21 dist-test-slave-4wm5 krb5kdc[17277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328201, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:21.881245 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.858960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:57133 (local address 127.3.82.62:37813)
0925 09:23:21.859354 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.859358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.860551 (+ 1193us) server_negotiation.cc:407] Connection header received
0925 09:23:21.862677 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.862683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.862768 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.862874 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:21.865133 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.866183 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.867998 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.868437 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.868592 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.873038 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:21.873044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:21.873048 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:21.873087 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:21.877295 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:21.878633 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:21.878639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:21.878641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:21.878775 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:21.879265 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:21.879270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:21.879273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:21.879592 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:21.879764 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.880581 (+ 817us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.880893 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260925 09:23:21.882416 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.859533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:57133)
0925 09:23:21.860346 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:21.860394 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:21.862297 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:21.863235 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:21.863246 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:21.864076 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:21.864900 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.864924 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.866456 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.866461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.867812 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.867825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.868693 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.868700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.868963 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.869787 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:21.869819 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:21.872741 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:21.877476 (+ 4735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:21.877487 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:21.877509 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:21.878354 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:21.878901 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:21.878909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:21.878913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:21.879128 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:21.879735 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:21.879745 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:21.880147 (+ 402us) client_negotiation.cc:769] Sending connection context
0925 09:23:21.880426 (+ 279us) client_negotiation.cc:240] Negotiation successful
0925 09:23:21.881042 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":329,"threads_started":1}
I20260925 09:23:21.884143 18313 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37813
I20260925 09:23:21.884510 18313 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:21.884537 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.865866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.3.82.62:37813)
0925 09:23:21.866617 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.866641 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.867265 (+ 624us) server_negotiation.cc:407] Connection header received
0925 09:23:21.867469 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.867480 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.867559 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.867679 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:21.869498 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.870914 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.872910 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.873451 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.873552 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.877457 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:21.877465 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:21.877469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:21.877522 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:21.881051 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:21.882156 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:21.882162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:21.882166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:21.882315 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:21.882828 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:21.882835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:21.882844 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:21.883165 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:21.883491 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.883864 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.884143 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":385,"threads_started":1}
I20260925 09:23:21.885499 18313 heartbeater.cc:511] Master 127.3.82.62:37813 requested a full tablet report, sending...
I20260925 09:23:21.887498 17864 ts_manager.cc:194] Registered new tserver with Master: 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069)
I20260925 09:23:21.888577 17864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:57133
I20260925 09:23:21.895593 18050 heartbeater.cc:503] Master 127.3.82.62:37813 was elected leader, sending a full tablet report...
I20260925 09:23:21.901114 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:23:21.909119 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.902293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.3.82.62:37813)
0925 09:23:21.902505 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.902509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.902953 (+ 444us) server_negotiation.cc:407] Connection header received
0925 09:23:21.903182 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.903188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.903362 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.903503 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:23:21.904056 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.904797 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.906751 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.907228 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.907337 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.907919 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:23:21.908483 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:23:21.908604 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.908806 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.908883 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 09:23:21.910324 17864 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:23:21.918114 17864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:23:21.959363 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.933103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.3.82.1:38121)
0925 09:23:21.945446 (+ 12343us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.945456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.945900 (+ 444us) server_negotiation.cc:407] Connection header received
0925 09:23:21.946352 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.946364 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.946665 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.946947 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:21.948311 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.950974 (+ 2663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.955730 (+ 4756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.956888 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.957090 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.957411 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.958730 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.958950 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12140,"thread_start_us":11948,"threads_started":1}
I20260925 09:23:21.959872 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.932620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37322 (local address 127.3.82.3:44069)
0925 09:23:21.945036 (+ 12416us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.945051 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.945106 (+ 55us) server_negotiation.cc:407] Connection header received
0925 09:23:21.945260 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.945274 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.945781 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.946129 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:21.947573 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.950133 (+ 2560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.957072 (+ 6939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.958505 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.958687 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.958946 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.959270 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.959446 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12145,"thread_start_us":11942,"threads_started":1}
I20260925 09:23:21.959965 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.932409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44069 (local address 127.0.0.1:37322)
0925 09:23:21.943877 (+ 11468us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:21.943948 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:21.944273 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:21.946573 (+ 2300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:21.946587 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:21.946636 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:21.947243 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.947262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.950471 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.951707 (+ 1236us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.955388 (+ 3681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.955407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.958759 (+ 3352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.958766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.959060 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.959087 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 09:23:21.959220 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:23:21.959459 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11025,"mutex_wait_us":115,"thread_start_us":13049,"threads_started":1}
I20260925 09:23:21.960695 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.932672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:38121 (local address 127.0.0.1:46574)
0925 09:23:21.945728 (+ 13056us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:21.945745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:21.945983 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:21.947333 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:21.947343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:21.947387 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:21.947986 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.948003 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.951233 (+ 3230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.951244 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.955425 (+ 4181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.955443 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.958156 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.958165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.958418 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.958480 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 09:23:21.959920 (+ 1440us) client_negotiation.cc:240] Negotiation successful
0925 09:23:21.960243 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12937,"mutex_wait_us":97,"thread_start_us":10571,"threads_started":1}
I20260925 09:23:21.962661 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.946192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:39257 (local address 127.0.0.1:53730)
0925 09:23:21.953494 (+ 7302us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:21.953525 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:21.953806 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:21.954746 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:21.954755 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:21.954786 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:21.955388 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.955407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.957000 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.957007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.959427 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:21.959446 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.961652 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.961661 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:21.961891 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.961927 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:23:21.962043 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 09:23:21.962200 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7122,"mutex_wait_us":4,"thread_start_us":6926,"threads_started":1}
I20260925 09:23:21.963436 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:21.946534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53730 (local address 127.3.82.2:39257)
0925 09:23:21.952870 (+ 6336us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:21.952879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:21.953654 (+ 775us) server_negotiation.cc:407] Connection header received
0925 09:23:21.954100 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:21.954110 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:21.954416 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:21.954664 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:21.955600 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.956830 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.959668 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:21.961367 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:21.961565 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:21.961775 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:21.962882 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0925 09:23:21.963120 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6137,"thread_start_us":6016,"threads_started":1}
I20260925 09:23:21.964891 18248 tablet_service.cc:1467] Processing CreateTablet for tablet db1e1f644c5648e1ba0d85b799c602a5 (DEFAULT_TABLE table=test-table [id=404aee476699478eb6fd357d17fa06c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:21.964941 17985 tablet_service.cc:1467] Processing CreateTablet for tablet db1e1f644c5648e1ba0d85b799c602a5 (DEFAULT_TABLE table=test-table [id=404aee476699478eb6fd357d17fa06c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:21.966554 18248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db1e1f644c5648e1ba0d85b799c602a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:21.966579 17985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db1e1f644c5648e1ba0d85b799c602a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:21.967733 18117 tablet_service.cc:1467] Processing CreateTablet for tablet db1e1f644c5648e1ba0d85b799c602a5 (DEFAULT_TABLE table=test-table [id=404aee476699478eb6fd357d17fa06c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:21.969389 18117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db1e1f644c5648e1ba0d85b799c602a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:21.982606 18329 tablet_bootstrap.cc:492] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Bootstrap starting.
I20260925 09:23:21.986850 18329 tablet_bootstrap.cc:654] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:21.988534 18331 tablet_bootstrap.cc:492] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Bootstrap starting.
I20260925 09:23:21.988593 18329 log.cc:824] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:21.990765 18330 tablet_bootstrap.cc:492] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Bootstrap starting.
I20260925 09:23:21.992547 18331 tablet_bootstrap.cc:654] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:21.993149 18329 tablet_bootstrap.cc:492] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: No bootstrap required, opened a new log
I20260925 09:23:21.993470 18329 ts_tablet_manager.cc:1397] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260925 09:23:21.994051 18331 log.cc:824] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:21.995303 18330 tablet_bootstrap.cc:654] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:21.996961 18330 log.cc:824] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:21.999164 18331 tablet_bootstrap.cc:492] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: No bootstrap required, opened a new log
I20260925 09:23:21.999536 18331 ts_tablet_manager.cc:1397] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260925 09:23:22.000236 18330 tablet_bootstrap.cc:492] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: No bootstrap required, opened a new log
I20260925 09:23:22.000541 18330 ts_tablet_manager.cc:1397] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260925 09:23:22.006078 18329 raft_consensus.cc:348] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.006845 18329 raft_consensus.cc:374] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:22.007042 18329 raft_consensus.cc:729] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478126b207d14c4a9ccf97201c4c96e8, State: Initialized, Role: FOLLOWER
I20260925 09:23:22.007943 18329 consensus_queue.cc:260] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.009348 18313 heartbeater.cc:503] Master 127.3.82.62:37813 was elected leader, sending a full tablet report...
I20260925 09:23:22.009735 18330 raft_consensus.cc:348] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.010393 18330 raft_consensus.cc:374] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:22.010563 18330 raft_consensus.cc:729] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db587fbeda384badb57c60114def46e5, State: Initialized, Role: FOLLOWER
I20260925 09:23:22.010985 18329 ts_tablet_manager.cc:1428] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Time spent starting tablet: real 0.017s user 0.017s sys 0.001s
I20260925 09:23:22.011471 18330 consensus_queue.cc:260] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.011641 18331 raft_consensus.cc:348] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.012223 18331 raft_consensus.cc:374] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:22.012377 18331 raft_consensus.cc:729] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c092ef305014c4dbd2176bc70917fe0, State: Initialized, Role: FOLLOWER
I20260925 09:23:22.013221 18330 ts_tablet_manager.cc:1428] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260925 09:23:22.014128 18331 consensus_queue.cc:260] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.016983 18331 ts_tablet_manager.cc:1428] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
I20260925 09:23:22.016984 18182 heartbeater.cc:503] Master 127.3.82.62:37813 was elected leader, sending a full tablet report...
W20260925 09:23:22.102720 18051 tablet.cc:2369] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:23:22.104339 18183 tablet.cc:2369] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:23:22.107801 18314 tablet.cc:2369] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:22.221546 18336 raft_consensus.cc:482] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:23:22.222004 18336 raft_consensus.cc:504] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.224081 18336 leader_election.cc:288] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069), 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257)
I20260925 09:23:22.231128 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.224646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:41985 (local address 127.3.82.3:44069)
0925 09:23:22.224868 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:22.224872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:22.225360 (+ 488us) server_negotiation.cc:407] Connection header received
0925 09:23:22.225546 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:22.225551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:22.225650 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:22.225781 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:22.226697 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.227435 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.229264 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.230158 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.230325 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.230439 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:22.230716 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 09:23:22.230847 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:23:22.231297 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.224477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:44069 (local address 127.3.82.1:41985)
0925 09:23:22.225192 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:22.225215 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:22.225376 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:22.225863 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:22.225871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:22.225909 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:22.226538 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:22.226551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.227577 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.227583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:22.229081 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:22.229093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.230382 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.230388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:22.230551 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.230576 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:23:22.230686 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:23:22.230841 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":22,"thread_start_us":370,"threads_started":1}
I20260925 09:23:22.231969 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.225858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:36335 (local address 127.3.82.2:39257)
0925 09:23:22.226063 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:22.226094 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:22.226668 (+ 574us) server_negotiation.cc:407] Connection header received
0925 09:23:22.226874 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:22.226880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:22.226948 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:22.227026 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:22.227662 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.228340 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.230148 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.230919 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.231082 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.231185 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:22.231578 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 09:23:22.231710 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 09:23:22.232098 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.225397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:39257 (local address 127.3.82.1:36335)
0925 09:23:22.226549 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:22.226565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:22.226726 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:22.227158 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:22.227166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:22.227192 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:22.227499 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:22.227506 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.228482 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.228487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:22.229974 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:22.229993 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.231167 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.231173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:22.231404 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.231437 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:23:22.231549 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 09:23:22.231680 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":1,"thread_start_us":840,"threads_started":1}
I20260925 09:23:22.232731 18268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1e1f644c5648e1ba0d85b799c602a5" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478126b207d14c4a9ccf97201c4c96e8" is_pre_election: true
I20260925 09:23:22.233312 18268 raft_consensus.cc:2454] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 0.
I20260925 09:23:22.233253 18137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1e1f644c5648e1ba0d85b799c602a5" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c092ef305014c4dbd2176bc70917fe0" is_pre_election: true
I20260925 09:23:22.233709 18137 raft_consensus.cc:2454] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 0.
I20260925 09:23:22.234097 17940 leader_election.cc:302] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478126b207d14c4a9ccf97201c4c96e8, db587fbeda384badb57c60114def46e5; no voters:
I20260925 09:23:22.234498 18336 raft_consensus.cc:2781] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:23:22.234699 18336 raft_consensus.cc:482] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:23:22.234841 18336 raft_consensus.cc:3037] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:22.237509 18336 raft_consensus.cc:504] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.238294 18336 leader_election.cc:288] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 election: Requested vote from peers 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069), 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257)
I20260925 09:23:22.239113 18137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1e1f644c5648e1ba0d85b799c602a5" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c092ef305014c4dbd2176bc70917fe0"
I20260925 09:23:22.239113 18268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1e1f644c5648e1ba0d85b799c602a5" candidate_uuid: "db587fbeda384badb57c60114def46e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478126b207d14c4a9ccf97201c4c96e8"
I20260925 09:23:22.239387 18137 raft_consensus.cc:3037] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:22.239444 18268 raft_consensus.cc:3037] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:22.243233 18137 raft_consensus.cc:2454] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 1.
I20260925 09:23:22.243234 18268 raft_consensus.cc:2454] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db587fbeda384badb57c60114def46e5 in term 1.
I20260925 09:23:22.243919 17940 leader_election.cc:302] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478126b207d14c4a9ccf97201c4c96e8, db587fbeda384badb57c60114def46e5; no voters:
I20260925 09:23:22.244320 18336 raft_consensus.cc:2781] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:22.244951 18336 raft_consensus.cc:686] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 1 LEADER]: Becoming Leader. State: Replica: db587fbeda384badb57c60114def46e5, State: Running, Role: LEADER
I20260925 09:23:22.245608 18336 consensus_queue.cc:237] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } }
I20260925 09:23:22.251284 17864 catalog_manager.cc:5495] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to db587fbeda384badb57c60114def46e5 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "db587fbeda384badb57c60114def46e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db587fbeda384badb57c60114def46e5" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 38121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 } health_report { overall_health: UNKNOWN } } }
I20260925 09:23:22.338068 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.331956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.3.82.1:38121)
0925 09:23:22.332208 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:22.332214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:22.332355 (+ 141us) server_negotiation.cc:407] Connection header received
0925 09:23:22.332638 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:22.332642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:22.332761 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:22.332934 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:23:22.333322 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.334153 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.335548 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.335960 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.336103 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.336657 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:23:22.337371 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:23:22.337562 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:22.337641 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 09:23:22.337730 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 09:23:22.347026 18137 raft_consensus.cc:1261] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Refusing update from remote peer db587fbeda384badb57c60114def46e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:22.347218 18268 raft_consensus.cc:1261] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Refusing update from remote peer db587fbeda384badb57c60114def46e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:22.348489 18340 consensus_queue.cc:1035] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c092ef305014c4dbd2176bc70917fe0" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:22.348946 18336 consensus_queue.cc:1035] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478126b207d14c4a9ccf97201c4c96e8" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 44069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:22.379565 18344 mvcc.cc:205] Tried to move back new op lower bound from 7333184316802560000 to 7333184316403326976. Current Snapshot: MvccSnapshot[applied={T|T < 7333184316802560000}]
I20260925 09:23:22.387615 17865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:22.387939 17865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:22.391651 17865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Sending DeleteTablet for 3 replicas of tablet db1e1f644c5648e1ba0d85b799c602a5
I20260925 09:23:22.393024 18117 tablet_service.cc:1514] Processing DeleteTablet for tablet db1e1f644c5648e1ba0d85b799c602a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:22 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:53730
I20260925 09:23:22.393042 17985 tablet_service.cc:1514] Processing DeleteTablet for tablet db1e1f644c5648e1ba0d85b799c602a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:22 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:46574
I20260925 09:23:22.393280 18248 tablet_service.cc:1514] Processing DeleteTablet for tablet db1e1f644c5648e1ba0d85b799c602a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:22 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:37322
I20260925 09:23:22.394269 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17902
I20260925 09:23:22.408885 18353 tablet_replica.cc:330] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: stopping tablet replica
I20260925 09:23:22.409147 18354 tablet_replica.cc:330] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: stopping tablet replica
I20260925 09:23:22.409503 18354 raft_consensus.cc:2229] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:22.409492 18353 raft_consensus.cc:2229] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:23:22.410099 18354 raft_consensus.cc:2258] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:22.410243 18353 raft_consensus.cc:2258] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:22.411067 18352 tablet_replica.cc:330] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: stopping tablet replica
I20260925 09:23:22.411515 18352 raft_consensus.cc:2229] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:22.412048 18352 raft_consensus.cc:2258] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:22.414186 18353 ts_tablet_manager.cc:1905] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:22.414395 18354 ts_tablet_manager.cc:1905] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:22.416549 18352 ts_tablet_manager.cc:1905] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:22.424633 18353 ts_tablet_manager.cc:1918] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:22.424921 18353 log.cc:1196] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/db1e1f644c5648e1ba0d85b799c602a5
I20260925 09:23:22.425716 18353 ts_tablet_manager.cc:1939] T db1e1f644c5648e1ba0d85b799c602a5 P db587fbeda384badb57c60114def46e5: Deleting consensus metadata
I20260925 09:23:22.428254 17852 catalog_manager.cc:4841] TS db587fbeda384badb57c60114def46e5 (127.3.82.1:38121): tablet db1e1f644c5648e1ba0d85b799c602a5 (table test-table [id=404aee476699478eb6fd357d17fa06c7]) successfully deleted
I20260925 09:23:22.428668 18352 ts_tablet_manager.cc:1918] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:22.428895 18352 log.cc:1196] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/db1e1f644c5648e1ba0d85b799c602a5
I20260925 09:23:22.429570 18352 ts_tablet_manager.cc:1939] T db1e1f644c5648e1ba0d85b799c602a5 P 7c092ef305014c4dbd2176bc70917fe0: Deleting consensus metadata
I20260925 09:23:22.431933 17851 catalog_manager.cc:4841] TS 7c092ef305014c4dbd2176bc70917fe0 (127.3.82.2:39257): tablet db1e1f644c5648e1ba0d85b799c602a5 (table test-table [id=404aee476699478eb6fd357d17fa06c7]) successfully deleted
I20260925 09:23:22.433586 18354 ts_tablet_manager.cc:1918] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:22.433832 18354 log.cc:1196] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/db1e1f644c5648e1ba0d85b799c602a5
I20260925 09:23:22.434432 18354 ts_tablet_manager.cc:1939] T db1e1f644c5648e1ba0d85b799c602a5 P 478126b207d14c4a9ccf97201c4c96e8: Deleting consensus metadata
I20260925 09:23:22.436412 17851 catalog_manager.cc:4841] TS 478126b207d14c4a9ccf97201c4c96e8 (127.3.82.3:44069): tablet db1e1f644c5648e1ba0d85b799c602a5 (table test-table [id=404aee476699478eb6fd357d17fa06c7]) successfully deleted
I20260925 09:23:22.448788 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.395493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.3.82.1:38121)
0925 09:23:22.395684 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:22.395690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:22.430435 (+ 34745us) server_negotiation.cc:407] Connection header received
0925 09:23:22.430655 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:22.430660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:22.430752 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:22.430858 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:22.432489 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.433735 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.435200 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.435916 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.436116 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.440010 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:22.440035 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:22.440059 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:22.440112 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:22.444920 (+ 4808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:22.446123 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:22.446135 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:22.446139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:22.446373 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:22.446877 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:22.446882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:22.446886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:22.447414 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:22.447785 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:22.448085 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 09:23:22.448366 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:23:22.449784 18045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:22.557219 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17902
I20260925 09:23:22.575949 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 18054
I20260925 09:23:22.591991 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.576859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53736 (local address 127.3.82.2:39257)
0925 09:23:22.577094 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:22.577099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:22.577119 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:23:22.577228 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:22.577233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:22.577317 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:22.577416 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:22.579188 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.579959 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.581676 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.582380 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.582488 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.585401 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:22.585419 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:22.585435 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:22.585475 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:22.588694 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:22.589662 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:22.589670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:22.589672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:22.589921 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:22.590472 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:22.590476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:22.590478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:22.590914 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:22.591129 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:22.591501 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 09:23:22.591677 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:23:22.592752 18177 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:22.688022 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 18054
I20260925 09:23:22.705763 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 18185
I20260925 09:23:22.722589 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:22.706723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.3.82.3:44069)
0925 09:23:22.706896 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:22.706900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:22.706919 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:23:22.707172 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:22.707177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:22.707245 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:22.707337 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:22.709000 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.709812 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.711545 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:22.711951 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:22.712061 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:22.715330 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:22.715349 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:22.715366 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:22.715426 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:22.719155 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:22.720121 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:22.720131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:22.720134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:22.720325 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:22.720801 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:22.720805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:22.720807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:22.721303 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:22.721539 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:22.722107 (+ 568us) server_negotiation.cc:299] Negotiation successful
0925 09:23:22.722315 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:23:22.723215 18308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:22.815531 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 18185
I20260925 09:23:22.832926 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 17832
I20260925 09:23:22.834259 17895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:22.950767 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 17832
I20260925 09:23:22.973387 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37813
--webserver_interface=127.3.82.62
--webserver_port=45477
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:23.325489 18360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:23.325819 18360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:23.326004 18360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:23.333666 18360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:23.333797 18360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:23.333932 18360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:23.333999 18360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:23.334038 18360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:23.344321 18360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=45477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:23.345705 18360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:23.347250 18360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:23.360508 18365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:23.360391 18368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:23.361011 18366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:23.361867 18360 server_base.cc:1034] running on GCE node
I20260925 09:23:23.362767 18360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:23.364349 18360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:23.365631 18360 hybrid_clock.cc:648] HybridClock initialized: now 1790328203365554 us; error 71 us; skew 500 ppm
I20260925 09:23:23.366114 18360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:23 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:23.369709 18360 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:23.371411 18360 webserver.cc:460] Webserver started at http://127.3.82.62:45477/ using document root <none> and password file <none>
I20260925 09:23:23.372148 18360 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:23.372303 18360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:23.378473 18360 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:23:23.383909 18375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:23.385589 18360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:23:23.385924 18360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2833e06b4a584d7b9373387c85ed8056"
format_stamp: "Formatted at 2026-09-25 09:23:16 on dist-test-slave-4wm5"
server_key: "d6e8933a43654a738ee61e5a29360c1a"
server_key_iv: "b6e7eca6e53dedba9a708b8f4c41cdf1"
server_key_version: "encryptionkey@0"
I20260925 09:23:23.387061 18360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:23.406481 18360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:23.411806 18360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:23.412187 18360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:23.434425 18360 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37813
I20260925 09:23:23.434466 18426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37813 every 8 connection(s)
I20260925 09:23:23.436115 18360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:23.439215 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18360
I20260925 09:23:23.440032 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:38121
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:37813
--webserver_port=36781
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:23.445022 18427 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:23:23.451200 18427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap starting.
I20260925 09:23:23.487254 18427 log.cc:824] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:23.511613 18427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:23:23.512202 18427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056: Bootstrap complete.
I20260925 09:23:23.522140 18427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:23.523109 18427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2833e06b4a584d7b9373387c85ed8056, State: Initialized, Role: FOLLOWER
I20260925 09:23:23.523816 18427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:23.524072 18427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:23:23.524200 18427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:23:23.524390 18427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:23:23.529036 18427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:23.529619 18427 leader_election.cc:302] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2833e06b4a584d7b9373387c85ed8056; no voters:
I20260925 09:23:23.530368 18427 leader_election.cc:288] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 09:23:23.530601 18431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 09:23:23.532195 18431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [term 3 LEADER]: Becoming Leader. State: Replica: 2833e06b4a584d7b9373387c85ed8056, State: Running, Role: LEADER
I20260925 09:23:23.532861 18427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:23.532910 18431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } }
I20260925 09:23:23.537186 18432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2833e06b4a584d7b9373387c85ed8056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } } }
I20260925 09:23:23.537621 18432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:23.538465 18433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2833e06b4a584d7b9373387c85ed8056. Latest consensus state: current_term: 3 leader_uuid: "2833e06b4a584d7b9373387c85ed8056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2833e06b4a584d7b9373387c85ed8056" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37813 } } }
I20260925 09:23:23.538760 18433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2833e06b4a584d7b9373387c85ed8056 [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:23.551309 18438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:23.563721 18438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:23.565487 18438 catalog_manager.cc:1259] Loaded cluster ID: eeeb1a5bfc8145eaa682b41ed5082793
I20260925 09:23:23.565701 18438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 09:23:23.568078 18438 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3294e679ad google::LogMessage::Fail() at ??:0
@ 0x7f3294e6a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3294e674de google::LogMessage::Flush() at ??:0
@ 0x7f3294e68a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f32aa89e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
I20260925 09:23:23.599007 18451 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f32aa89b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f32aa923610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f32aa923439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3297ab3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3297ab691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3297ab6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3297a91eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f329ad6b6db start_thread at ??:0
@ 0x7f3293d0161f clone at ??:0
W20260925 09:23:23.899823 18429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:23.900156 18429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:23.900307 18429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:23.908206 18429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:23.908336 18429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:23.908525 18429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:23:23.919456 18429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:38121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=36781
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:23.920928 18429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:23.922360 18429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:23.937017 18470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:23.937165 18472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:23.937626 18469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:23.938146 18429 server_base.cc:1034] running on GCE node
I20260925 09:23:23.938868 18429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:23.939983 18429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:23.941241 18429 hybrid_clock.cc:648] HybridClock initialized: now 1790328203941164 us; error 64 us; skew 500 ppm
I20260925 09:23:23.941601 18429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:23 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328203, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:23.945043 18429 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:23:23.946866 18429 webserver.cc:460] Webserver started at http://127.3.82.1:36781/ using document root <none> and password file <none>
I20260925 09:23:23.947685 18429 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:23.947870 18429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:23.953935 18429 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:23.957366 18479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:23.959234 18429 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:23.959484 18429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "db587fbeda384badb57c60114def46e5"
format_stamp: "Formatted at 2026-09-25 09:23:17 on dist-test-slave-4wm5"
server_key: "c9d53d87660cf4a3a8e8316e82d7634e"
server_key_iv: "5fcbdd6dd85b7e0081edc2c37c8eb5e3"
server_key_version: "encryptionkey@0"
I20260925 09:23:23.960729 18429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:23.980779 18429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:23.986326 18429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:23.986673 18429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:23.987869 18429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:23.989436 18429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:23.989548 18429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:23.989691 18429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:23.989806 18429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:24.027662 18429 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:38121
I20260925 09:23:24.027741 18591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:38121 every 8 connection(s)
I20260925 09:23:24.029616 18429 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:23:24.032886 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18429
I20260925 09:23:24.033600 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:39257
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:37813
--webserver_port=37151
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:24.035697 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:24.033365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:52961)
0925 09:23:24.034364 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:24.034642 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":447,"threads_started":1}
W20260925 09:23:24.036670 18592 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:37813 (0 consecutive failures): Network error: Failed to ping master at 127.3.82.62:37813: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
I20260925 09:23:24.037909 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:24.037402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:45427)
0925 09:23:24.037610 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:24.037702 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260925 09:23:24.039042 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:24.038592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:58695)
0925 09:23:24.038824 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:24.038910 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260925 09:23:24.039295 18592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:23:24.405441 18595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:24.405794 18595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:24.405968 18595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:24.413362 18595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:24.413481 18595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:24.413602 18595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:23:24.423956 18595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:39257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=37151
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:24.425223 18595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:24.426542 18595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:24.441440 18600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:24.441337 18603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:24.441365 18601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:24.442446 18595 server_base.cc:1034] running on GCE node
I20260925 09:23:24.443188 18595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:24.444271 18595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:24.445554 18595 hybrid_clock.cc:648] HybridClock initialized: now 1790328204445487 us; error 64 us; skew 500 ppm
I20260925 09:23:24.446002 18595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:24 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328204, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:24.449607 18595 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:23:24.451526 18595 webserver.cc:460] Webserver started at http://127.3.82.2:37151/ using document root <none> and password file <none>
I20260925 09:23:24.452363 18595 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:24.452531 18595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:24.459087 18595 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:23:24.462580 18610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:24.464318 18595 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:23:24.464550 18595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "7c092ef305014c4dbd2176bc70917fe0"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "3d52e23de0ce588150f7b53631370f1e"
server_key_iv: "b6f1bbf105e62f65d8fc75b74fafdd69"
server_key_version: "encryptionkey@0"
I20260925 09:23:24.465803 18595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:24.489641 18595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:24.494952 18595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:24.495289 18595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:24.496378 18595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:24.497862 18595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:24.497997 18595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:24.498163 18595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:24.498253 18595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:24.533890 18595 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:39257
I20260925 09:23:24.534019 18722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:39257 every 8 connection(s)
I20260925 09:23:24.535419 18595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:23:24.540625 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:24.538656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:58597)
0925 09:23:24.539489 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:24.539738 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":296,"threads_started":1}
W20260925 09:23:24.541339 18723 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:37813 (0 consecutive failures): Network error: Failed to ping master at 127.3.82.62:37813: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
I20260925 09:23:24.542503 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:24.541971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:34035)
0925 09:23:24.542190 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:24.542336 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 09:23:24.542680 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18595
I20260925 09:23:24.543376 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:44069
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:37813
--webserver_port=44805
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:23:24.543706 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:24.543219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:43859)
0925 09:23:24.543392 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:24.543494 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 09:23:24.544016 18723 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:23:24.883869 18726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:24.884202 18726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:24.884379 18726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:24.892786 18726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:24.892923 18726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:24.893114 18726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:23:24.903489 18726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:44069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=44805
--tserver_master_addrs=127.3.82.62:37813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:24.904906 18726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:24.906411 18726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:24.922048 18732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:24.922169 18731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:24.922683 18734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:24.923137 18726 server_base.cc:1034] running on GCE node
I20260925 09:23:24.923740 18726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:24.924813 18726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:24.926137 18726 hybrid_clock.cc:648] HybridClock initialized: now 1790328204926075 us; error 54 us; skew 500 ppm
I20260925 09:23:24.926541 18726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:24 dist-test-slave-4wm5 krb5kdc[17277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328204, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:24.930148 18726 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:23:24.931890 18726 webserver.cc:460] Webserver started at http://127.3.82.3:44805/ using document root <none> and password file <none>
I20260925 09:23:24.932646 18726 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:24.932796 18726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:24.938881 18726 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:23:24.942024 18741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:24.943646 18726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:23:24.943874 18726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "478126b207d14c4a9ccf97201c4c96e8"
format_stamp: "Formatted at 2026-09-25 09:23:18 on dist-test-slave-4wm5"
server_key: "345cb129ebf94f5b39d13feb83287648"
server_key_iv: "4393914df92673011e09f934d481eb6f"
server_key_version: "encryptionkey@0"
I20260925 09:23:24.945008 18726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:24.965489 18726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:24.970424 18726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:24.970728 18726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:24.971902 18726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:24.973309 18726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:24.973443 18726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:24.973584 18726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:24.973672 18726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:25.010154 18726 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:44069
I20260925 09:23:25.010258 18853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:44069 every 8 connection(s)
I20260925 09:23:25.011744 18726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:23:25.017293 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:25.015190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:57089)
0925 09:23:25.016031 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:25.016247 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":350,"threads_started":1}
W20260925 09:23:25.018157 18854 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:37813 (0 consecutive failures): Network error: Failed to ping master at 127.3.82.62:37813: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
I20260925 09:23:25.019009 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18726
I20260925 09:23:25.019260 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:25.018827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:36463)
0925 09:23:25.019014 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:25.019113 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 09:23:25.020257 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:25.019869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:34519)
0925 09:23:25.020053 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:25.020135 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260925 09:23:25.020547 18854 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 09:23:25.041083 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:25.040161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:50337)
0925 09:23:25.040726 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:25.040829 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":290,"threads_started":1}
I20260925 09:23:25.545725 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:25.544743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:47295)
0925 09:23:25.545371 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:25.545469 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":379,"threads_started":1}
I20260925 09:23:26.022207 18860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:26.021262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:60335)
0925 09:23:26.021733 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:26.021886 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":240,"threads_started":1}
I20260925 09:23:26.043501 18861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:26.042551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:40917)
0925 09:23:26.043222 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:26.043326 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":342,"threads_started":1}
I20260925 09:23:26.547730 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:26.546939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:58721)
0925 09:23:26.547457 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:26.547545 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":270,"threads_started":1}
I20260925 09:23:27.024190 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:27.023212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:37633)
0925 09:23:27.023817 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:27.023950 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":277,"threads_started":1}
I20260925 09:23:27.045466 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:27.044655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:59309)
0925 09:23:27.045161 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:27.045265 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":267,"threads_started":1}
I20260925 09:23:27.549731 18865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:27.548894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:38301)
0925 09:23:27.549435 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:27.549534 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":254,"threads_started":1}
I20260925 09:23:28.026044 18866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:28.025218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:55269)
0925 09:23:28.025706 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:28.025791 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":249,"threads_started":1}
I20260925 09:23:28.047477 18867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:28.046499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:43127)
0925 09:23:28.047091 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:28.047297 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":322,"threads_started":1}
I20260925 09:23:28.551684 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:28.550863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:49767)
0925 09:23:28.551388 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:28.551477 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":295,"threads_started":1}
I20260925 09:23:29.028268 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:29.027330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:42553)
0925 09:23:29.027948 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:29.028042 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":292,"threads_started":1}
I20260925 09:23:29.049698 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:29.048710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:36827)
0925 09:23:29.049328 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:29.049492 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":378,"threads_started":1}
I20260925 09:23:29.553593 18871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:29.552826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:52779)
0925 09:23:29.553329 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:29.553414 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":265,"threads_started":1}
I20260925 09:23:30.030685 18872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:30.029503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:56703)
0925 09:23:30.030410 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:30.030491 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":479,"threads_started":1}
I20260925 09:23:30.051821 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:30.050889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:52821)
0925 09:23:30.051520 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:30.051647 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":406,"threads_started":1}
I20260925 09:23:30.555578 18874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:30.554751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:52289)
0925 09:23:30.555269 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:30.555394 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":254,"threads_started":1}
I20260925 09:23:31.032655 18875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:31.031761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:38829)
0925 09:23:31.032341 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:31.032457 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":262,"threads_started":1}
I20260925 09:23:31.053951 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:31.052924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:59153)
0925 09:23:31.053593 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:31.053683 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":349,"threads_started":1}
I20260925 09:23:31.557806 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:31.556803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:35417)
0925 09:23:31.557452 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:31.557540 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":373,"threads_started":1}
I20260925 09:23:32.034761 18878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:32.033763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:49985)
0925 09:23:32.034442 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:32.034533 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":272,"threads_started":1}
I20260925 09:23:32.055950 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:32.054989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:58243)
0925 09:23:32.055644 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:32.055750 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":356,"threads_started":1}
I20260925 09:23:32.559677 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:32.558944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:40715)
0925 09:23:32.559400 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:32.559485 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":223,"threads_started":1}
I20260925 09:23:33.036849 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:33.035963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:43105)
0925 09:23:33.036512 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:33.036667 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":294,"threads_started":1}
I20260925 09:23:33.057989 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:33.057018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:39173)
0925 09:23:33.057635 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:33.057729 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":350,"threads_started":1}
I20260925 09:23:33.561417 18883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:33.560640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:34407)
0925 09:23:33.561140 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:33.561238 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":234,"threads_started":1}
I20260925 09:23:34.039064 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:34.038223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:47527)
0925 09:23:34.038783 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:34.038881 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":273,"threads_started":1}
I20260925 09:23:34.059871 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:34.059127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:45145)
0925 09:23:34.059618 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:34.059705 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":252,"threads_started":1}
I20260925 09:23:34.563285 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:34.562463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:35443)
0925 09:23:34.562951 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:34.563042 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":269,"threads_started":1}
I20260925 09:23:35.041428 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:35.040486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:43267)
0925 09:23:35.041090 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:35.041223 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":304,"threads_started":1}
I20260925 09:23:35.061950 18888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:35.060996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:57471)
0925 09:23:35.061568 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:35.061681 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":336,"threads_started":1}
I20260925 09:23:35.565361 18889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:35.564481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:47729)
0925 09:23:35.564984 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:35.565083 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":280,"threads_started":1}
I20260925 09:23:36.043658 18890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:36.042833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:56347)
0925 09:23:36.043339 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:36.043450 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":256,"threads_started":1}
I20260925 09:23:36.063932 18891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:36.062975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:37119)
0925 09:23:36.063648 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:36.063750 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":431,"threads_started":1}
I20260925 09:23:36.567333 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:36.566526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:51509)
0925 09:23:36.566984 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:36.567151 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":242,"threads_started":1}
I20260925 09:23:37.046046 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:37.044950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:54215)
0925 09:23:37.045656 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:37.045768 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":413,"threads_started":1}
I20260925 09:23:37.066206 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:37.065364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:47957)
0925 09:23:37.065919 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:37.066009 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":219,"threads_started":1}
I20260925 09:23:37.569334 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:37.568440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:49973)
0925 09:23:37.568991 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:37.569098 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":264,"threads_started":1}
I20260925 09:23:38.048307 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:38.047409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:52079)
0925 09:23:38.047985 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:38.048088 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":260,"threads_started":1}
I20260925 09:23:38.068166 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:38.067369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:58151)
0925 09:23:38.067871 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:38.067961 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":257,"threads_started":1}
I20260925 09:23:38.571201 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:38.570461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:33469)
0925 09:23:38.570926 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:38.571009 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":217,"threads_started":1}
I20260925 09:23:39.050323 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:39.049342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:50901)
0925 09:23:39.049954 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:39.050047 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":297,"threads_started":1}
I20260925 09:23:39.070731 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:39.069905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.1:42477)
0925 09:23:39.070444 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:39.070538 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":314,"threads_started":1}
I20260925 09:23:39.573354 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:39.572623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.2:35821)
0925 09:23:39.573065 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:39.573157 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":228,"threads_started":1}
I20260925 09:23:40.025481 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 18429
I20260925 09:23:40.052388 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:40.051619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37813 (local address 127.3.82.3:49599)
0925 09:23:40.052093 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:40.052186 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":261,"threads_started":1}
I20260925 09:23:40.063403 18902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:40.026931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43626 (local address 127.3.82.1:38121)
0925 09:23:40.027522 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:40.027547 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:40.027777 (+ 230us) server_negotiation.cc:407] Connection header received
0925 09:23:40.029630 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:40.029641 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:40.030184 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:40.030517 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:40.034430 (+ 3913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:40.035918 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:40.037359 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:40.038325 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:40.038482 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:40.055773 (+ 17291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:40.055783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:40.055802 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:40.055853 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:40.059092 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:40.060176 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:40.060185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:40.060193 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:40.060438 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:40.060927 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:40.060932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:40.060935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:40.061733 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:40.062126 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:40.062500 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 09:23:40.062985 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
I20260925 09:23:40.065074 18587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:40.155210 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 18429
I20260925 09:23:40.174147 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 18595
I20260925 09:23:40.194988 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:40.175481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.3.82.2:39257)
0925 09:23:40.176018 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:40.176033 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:40.176072 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:23:40.177571 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:40.177581 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:40.178301 (+ 720us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:40.178622 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:40.180151 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:40.181546 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:40.183134 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:40.183895 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:40.184039 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:40.187168 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:40.187177 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:40.187194 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:40.187243 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:40.190745 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:40.191580 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:40.191586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:40.191595 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:40.191916 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:40.192442 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:40.192446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:40.192448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:40.193095 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:40.193351 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:40.194190 (+ 839us) server_negotiation.cc:299] Negotiation successful
0925 09:23:40.194687 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":263,"threads_started":1}
I20260925 09:23:40.196579 18718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:40.287539 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 18595
I20260925 09:23:40.305248 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 18726
I20260925 09:23:40.324019 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:40.306330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.3.82.3:44069)
0925 09:23:40.306910 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:40.306924 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:40.306966 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:23:40.308629 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:40.308639 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:40.309076 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:40.309391 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:40.310876 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:40.312182 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:40.313516 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:40.314345 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:40.314495 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:40.317158 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:40.317166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:40.317184 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:40.317227 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:40.320447 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:40.321243 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:40.321250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:40.321257 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:40.321496 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:40.321948 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:40.321952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:40.321954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:40.322561 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:40.322798 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:40.323192 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 09:23:40.323690 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":262,"threads_started":1}
I20260925 09:23:40.325651 18849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:40.412438 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 18726
2026-09-25T09:23:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25198 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:23:40 dist-test-slave-4wm5 krb5kdc[18912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:23:40 dist-test-slave-4wm5 krb5kdc[18912](info): set up 2 sockets
Sep 25 09:23:40 dist-test-slave-4wm5 krb5kdc[18912](info): commencing operation
krb5kdc: starting...
W20260925 09:23:41.586668 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:23:41 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:23:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:23:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:23:41.855186 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37625
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:42.202450 18925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:42.202826 18925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:42.203002 18925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:42.210584 18925 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 09:23:42.210757 18925 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 09:23:42.210829 18925 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 09:23:42.210906 18925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:23:42.210986 18925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:42.211091 18925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:42.211166 18925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:23:42.211246 18925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:23:42.221401 18925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:42.222854 18925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:42.224269 18925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:42.237715 18931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:42.237860 18930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:42.237767 18933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:42.238720 18925 server_base.cc:1034] running on GCE node
I20260925 09:23:42.239614 18925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:42.241375 18925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:42.242700 18925 hybrid_clock.cc:648] HybridClock initialized: now 1790328222242628 us; error 41 us; skew 500 ppm
I20260925 09:23:42.243141 18925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:42 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328222, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:42.247305 18925 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:23:42.249200 18925 webserver.cc:460] Webserver started at http://127.3.82.62:40223/ using document root <none> and password file <none>
I20260925 09:23:42.250118 18925 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:42.250216 18925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:42.250547 18925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:42.253942 18925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "bc335470a63f48318196a52ffed6341d"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "e4686f6c4fb9334c70f49ed5986ccd67"
server_key_iv: "33cee414889439be871f2a68ab0ee2a1"
server_key_version: "encryptionkey@0"
I20260925 09:23:42.254912 18925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "bc335470a63f48318196a52ffed6341d"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "e4686f6c4fb9334c70f49ed5986ccd67"
server_key_iv: "33cee414889439be871f2a68ab0ee2a1"
server_key_version: "encryptionkey@0"
I20260925 09:23:42.261088 18925 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260925 09:23:42.265772 18940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:42.267640 18925 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:42.267936 18925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "bc335470a63f48318196a52ffed6341d"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "e4686f6c4fb9334c70f49ed5986ccd67"
server_key_iv: "33cee414889439be871f2a68ab0ee2a1"
server_key_version: "encryptionkey@0"
I20260925 09:23:42.268337 18925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:42.292881 18925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:42.298236 18925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:42.298599 18925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:42.320351 18925 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37625
I20260925 09:23:42.320396 18991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37625 every 8 connection(s)
I20260925 09:23:42.322167 18925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:23:42.326305 18992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:42.331759 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 18925
I20260925 09:23:42.332010 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:23:42.332827 3400 external_mini_cluster.cc:1428] Setting key ce424546659319665adeb4ffb246e74d
I20260925 09:23:42.339151 18992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap starting.
I20260925 09:23:42.342581 18992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:42.343703 18992 log.cc:824] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:42.346951 18992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: No bootstrap required, opened a new log
Sep 25 09:23:42 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:42.355700 18992 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:23:42.356139 18992 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:42.356259 18992 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc335470a63f48318196a52ffed6341d, State: Initialized, Role: FOLLOWER
I20260925 09:23:42.357055 18992 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:23:42.357511 18992 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:23:42.357687 18992 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:23:42.357936 18992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:42.359186 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:42.335321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41510 (local address 127.3.82.62:37625)
0925 09:23:42.336285 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:42.336298 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:42.336345 (+ 47us) server_negotiation.cc:407] Connection header received
0925 09:23:42.338258 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:42.338286 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:42.338756 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:42.339216 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:42.340842 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:42.342294 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:42.343889 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:42.344673 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:42.344904 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:42.349888 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:42.349911 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:42.349929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:42.349984 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:42.353808 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:42.354809 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:42.354818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:42.354826 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:42.355052 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:42.355541 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:42.355549 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:42.355557 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:42.356319 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:42.356710 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:42.357238 (+ 528us) server_negotiation.cc:299] Negotiation successful
0925 09:23:42.357800 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":433,"threads_started":1}
I20260925 09:23:42.361095 18992 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:23:42.361634 18992 leader_election.cc:302] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc335470a63f48318196a52ffed6341d; no voters:
I20260925 09:23:42.362322 18992 leader_election.cc:288] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:23:42.362496 18997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:42.363327 18997 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 LEADER]: Becoming Leader. State: Replica: bc335470a63f48318196a52ffed6341d, State: Running, Role: LEADER
I20260925 09:23:42.364099 18997 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:23:42.365010 18992 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:23:42.367079 18998 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc335470a63f48318196a52ffed6341d. Latest consensus state: current_term: 1 leader_uuid: "bc335470a63f48318196a52ffed6341d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } } }
I20260925 09:23:42.367487 18999 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc335470a63f48318196a52ffed6341d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } } }
I20260925 09:23:42.368007 18999 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:42.368040 18998 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: This master's current role is: LEADER
I20260925 09:23:42.374738 19005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:23:42.380126 19005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:23:42.390168 19005 catalog_manager.cc:1347] Generated new cluster ID: 10960b0e7b5349dbb028f475efaa6180
I20260925 09:23:42.390377 19005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:23:42.406803 19005 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:23:42.408439 19005 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:23:42.422936 19005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Generated new TSK 0
I20260925 09:23:42.424135 19005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:23:42.494840 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37625
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:42.850364 19018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:42.850708 19018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:42.850816 19018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:42.860170 19018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:42.860306 19018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:42.860436 19018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:23:42.872502 19018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:42.874138 19018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:42.875705 19018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:42.890318 19023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:42.890313 19024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:42.891702 19026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:42.892575 19018 server_base.cc:1034] running on GCE node
I20260925 09:23:42.893273 19018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:42.894445 19018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:42.895727 19018 hybrid_clock.cc:648] HybridClock initialized: now 1790328222895646 us; error 72 us; skew 500 ppm
I20260925 09:23:42.896173 19018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:42 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328222, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:42.900099 19018 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:23:42.901933 19018 webserver.cc:460] Webserver started at http://127.3.82.1:37329/ using document root <none> and password file <none>
I20260925 09:23:42.902707 19018 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:42.902858 19018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:42.903196 19018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:42.906380 19018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "aaac84088bd49f5d9bfcf9299bf2dadb"
server_key_iv: "08fc64156c4b716a03cab335e595d19d"
server_key_version: "encryptionkey@0"
I20260925 09:23:42.907193 19018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "aaac84088bd49f5d9bfcf9299bf2dadb"
server_key_iv: "08fc64156c4b716a03cab335e595d19d"
server_key_version: "encryptionkey@0"
I20260925 09:23:42.912885 19018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:23:42.917212 19033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:42.919004 19018 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:23:42.919323 19018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "aaac84088bd49f5d9bfcf9299bf2dadb"
server_key_iv: "08fc64156c4b716a03cab335e595d19d"
server_key_version: "encryptionkey@0"
I20260925 09:23:42.919576 19018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:42.942239 19018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:42.947530 19018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:42.947896 19018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:42.949047 19018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:42.950644 19018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:42.950752 19018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:42.950919 19018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:42.951011 19018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:42.989948 19018 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:43023
I20260925 09:23:42.990052 19145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:43023 every 8 connection(s)
I20260925 09:23:42.991392 19018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:23:42.992277 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19018
I20260925 09:23:42.992548 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:23:42.993396 3400 external_mini_cluster.cc:1428] Setting key 8086ae22a1feb577b1d6d303b1d8f0f1
Sep 25 09:23:43 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328222, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:43.015359 19148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:42.994762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:41093 (local address 127.3.82.62:37625)
0925 09:23:42.995306 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:42.995312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:42.996504 (+ 1192us) server_negotiation.cc:407] Connection header received
0925 09:23:42.998697 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:42.998703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:42.998799 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:42.998937 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:43.000635 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.001412 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.003333 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.003802 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.003929 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:43.008812 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:43.008818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:43.008821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:43.008863 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:43.011908 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:43.012801 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:43.012811 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:43.012813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:43.012943 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:43.013426 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:43.013437 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:43.013439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:43.013732 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:43.013929 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:43.014688 (+ 759us) server_negotiation.cc:299] Negotiation successful
0925 09:23:43.014934 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":273,"threads_started":1}
I20260925 09:23:43.016395 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:42.995495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:41093)
0925 09:23:42.996346 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:42.996392 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:42.998347 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:42.999218 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:42.999227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:42.999808 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:43.000437 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:43.000456 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.001590 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.001594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:43.003088 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:43.003110 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.004145 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.004153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:43.004547 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:43.005428 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:43.005475 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:43.008503 (+ 3028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:43.012104 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:43.012115 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:43.012136 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:43.012639 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:43.013072 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:43.013078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:43.013082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:43.013294 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:43.013893 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:43.013903 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:43.014292 (+ 389us) client_negotiation.cc:769] Sending connection context
0925 09:23:43.014589 (+ 297us) client_negotiation.cc:240] Negotiation successful
0925 09:23:43.015196 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":346,"threads_started":1}
I20260925 09:23:43.018426 19146 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37625
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:23:43.018828 19146 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:43.019611 19146 heartbeater.cc:511] Master 127.3.82.62:37625 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:23:43.022171 18957 ts_manager.cc:194] Registered new tserver with Master: 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023)
I20260925 09:23:43.023792 18957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:41093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:23:43.056550 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37625
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:43.404956 19152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:43.405331 19152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:43.405509 19152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:43.413239 19152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:43.413451 19152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:43.413651 19152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:23:43.425002 19152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:43.426527 19152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:43.427968 19152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:43.443051 19157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:43.443132 19160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:43.443197 19158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:43.444742 19152 server_base.cc:1034] running on GCE node
I20260925 09:23:43.445514 19152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:43.446885 19152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:43.448191 19152 hybrid_clock.cc:648] HybridClock initialized: now 1790328223448112 us; error 73 us; skew 500 ppm
I20260925 09:23:43.448601 19152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:43 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328223, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:43.452477 19152 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:23:43.454433 19152 webserver.cc:460] Webserver started at http://127.3.82.2:38907/ using document root <none> and password file <none>
I20260925 09:23:43.455209 19152 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:43.455329 19152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:43.455631 19152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:43.459128 19152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "0e16ceb5871241e5a95f693949484528"
format_stamp: "Formatted at 2026-09-25 09:23:43 on dist-test-slave-4wm5"
server_key: "02bdf4387ce49541578cddb73fe66451"
server_key_iv: "0bbbe861d6df931c48942f9c013f7ec6"
server_key_version: "encryptionkey@0"
I20260925 09:23:43.459971 19152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "0e16ceb5871241e5a95f693949484528"
format_stamp: "Formatted at 2026-09-25 09:23:43 on dist-test-slave-4wm5"
server_key: "02bdf4387ce49541578cddb73fe66451"
server_key_iv: "0bbbe861d6df931c48942f9c013f7ec6"
server_key_version: "encryptionkey@0"
I20260925 09:23:43.466720 19152 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260925 09:23:43.471673 19167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:43.473572 19152 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:23:43.473786 19152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "0e16ceb5871241e5a95f693949484528"
format_stamp: "Formatted at 2026-09-25 09:23:43 on dist-test-slave-4wm5"
server_key: "02bdf4387ce49541578cddb73fe66451"
server_key_iv: "0bbbe861d6df931c48942f9c013f7ec6"
server_key_version: "encryptionkey@0"
I20260925 09:23:43.474074 19152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:43.501715 19152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:43.507128 19152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:43.507481 19152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:43.508796 19152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:43.510432 19152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:43.510520 19152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:43.510671 19152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:43.510726 19152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:43.546519 19152 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:42383
I20260925 09:23:43.546648 19279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:42383 every 8 connection(s)
I20260925 09:23:43.547911 19152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:23:43.553352 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19152
I20260925 09:23:43.553543 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:23:43.554193 3400 external_mini_cluster.cc:1428] Setting key 2897de1256cebf6b7da6f79d15cc4e7b
Sep 25 09:23:43 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328223, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:43.572865 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:43.550814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:58989 (local address 127.3.82.62:37625)
0925 09:23:43.551460 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:43.551466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:43.552259 (+ 793us) server_negotiation.cc:407] Connection header received
0925 09:23:43.555184 (+ 2925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:43.555190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:43.555291 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:43.555431 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:43.557369 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.558941 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.560819 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.561287 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.561430 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:43.565682 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:43.565688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:43.565692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:43.565734 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:43.568927 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:43.569849 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:43.569856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:43.569860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:43.570009 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:43.570662 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:43.570666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:43.570669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:43.570997 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:43.571112 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:43.572167 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0925 09:23:43.572481 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":361,"threads_started":1}
I20260925 09:23:43.574290 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:43.551255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:58989)
0925 09:23:43.552018 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:43.552062 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:43.554867 (+ 2805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:43.555689 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:43.555699 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:43.556370 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:43.556994 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:43.557012 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.559311 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.559315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:43.560628 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:43.560642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:43.561540 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:43.561547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:43.561920 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:43.562671 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:43.562695 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:43.565372 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:43.569128 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:43.569138 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:43.569159 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:43.569672 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:43.570211 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:43.570219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:43.570225 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:43.570510 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:43.571182 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:43.571195 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:43.571711 (+ 516us) client_negotiation.cc:769] Sending connection context
0925 09:23:43.572050 (+ 339us) client_negotiation.cc:240] Negotiation successful
0925 09:23:43.572699 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":320,"threads_started":1}
I20260925 09:23:43.575893 19280 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37625
I20260925 09:23:43.576269 19280 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:43.577105 19280 heartbeater.cc:511] Master 127.3.82.62:37625 requested a full tablet report, sending...
I20260925 09:23:43.578852 18957 ts_manager.cc:194] Registered new tserver with Master: 0e16ceb5871241e5a95f693949484528 (127.3.82.2:42383)
I20260925 09:23:43.579777 18957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:58989
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:23:43.614971 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37625
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:23:43.971608 19286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:23:43.971954 19286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:23:43.972127 19286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:23:43.980247 19286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:23:43.980419 19286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:23:43.980621 19286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:23:43.992599 19286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:23:43.994402 19286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:23:43.996251 19286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:23:44.011387 19291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:44.011485 19292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:23:44.012089 19294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:23:44.012563 19286 server_base.cc:1034] running on GCE node
I20260925 09:23:44.013293 19286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:23:44.014621 19286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:23:44.015899 19286 hybrid_clock.cc:648] HybridClock initialized: now 1790328224015846 us; error 56 us; skew 500 ppm
I20260925 09:23:44.016407 19286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:23:44 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328224, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:23:44.020205 19286 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:23:44.021955 19286 webserver.cc:460] Webserver started at http://127.3.82.3:46765/ using document root <none> and password file <none>
I20260925 09:23:44.022735 19286 fs_manager.cc:359] Metadata directory not provided
I20260925 09:23:44.022876 19286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:23:44.023291 19286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:23:44.026678 19286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "1a0b6054639e4518bf894fa7ae3a61f4"
format_stamp: "Formatted at 2026-09-25 09:23:44 on dist-test-slave-4wm5"
server_key: "2d85f87becf97986e10d42140917aaec"
server_key_iv: "b3d70366660708a558b91ace2b3b64de"
server_key_version: "encryptionkey@0"
I20260925 09:23:44.027347 19146 heartbeater.cc:503] Master 127.3.82.62:37625 was elected leader, sending a full tablet report...
I20260925 09:23:44.027631 19286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "1a0b6054639e4518bf894fa7ae3a61f4"
format_stamp: "Formatted at 2026-09-25 09:23:44 on dist-test-slave-4wm5"
server_key: "2d85f87becf97986e10d42140917aaec"
server_key_iv: "b3d70366660708a558b91ace2b3b64de"
server_key_version: "encryptionkey@0"
I20260925 09:23:44.034536 19286 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:23:44.039047 19301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:44.040845 19286 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:23:44.041079 19286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "1a0b6054639e4518bf894fa7ae3a61f4"
format_stamp: "Formatted at 2026-09-25 09:23:44 on dist-test-slave-4wm5"
server_key: "2d85f87becf97986e10d42140917aaec"
server_key_iv: "b3d70366660708a558b91ace2b3b64de"
server_key_version: "encryptionkey@0"
I20260925 09:23:44.041380 19286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:23:44.065073 19286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:23:44.070171 19286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:23:44.070492 19286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:23:44.071694 19286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:23:44.073334 19286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:23:44.073457 19286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:44.073637 19286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:23:44.073714 19286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:23:44.108916 19286 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:46313
I20260925 09:23:44.109033 19413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:46313 every 8 connection(s)
I20260925 09:23:44.110328 19286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:23:44.111883 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19286
I20260925 09:23:44.112046 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:23:44.112495 3400 external_mini_cluster.cc:1428] Setting key 07afd251c6d353accb27683e233d80c6
Sep 25 09:23:44 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328224, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:23:44.135345 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.114839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:46647 (local address 127.3.82.62:37625)
0925 09:23:44.116070 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.116075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.116744 (+ 669us) server_negotiation.cc:407] Connection header received
0925 09:23:44.118792 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.118797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.118905 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.119028 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:44.120825 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.121601 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.123324 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.123796 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.123936 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.128651 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:44.128674 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:44.128680 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:44.128739 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:44.131714 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.132652 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.132657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.132659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.132809 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.133348 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.133352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.133355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.133690 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:44.133815 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.134785 (+ 970us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.135024 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1081,"thread_start_us":969,"threads_started":1}
I20260925 09:23:44.136234 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.115689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:46647)
0925 09:23:44.116588 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:44.116632 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:44.118489 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:44.119403 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:44.119412 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:44.119982 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:23:44.120627 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.120647 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.121778 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.121782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.123165 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.123177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.124030 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.124036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.124281 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.125400 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:23:44.125433 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:23:44.128388 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:23:44.131893 (+ 3505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:44.131901 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:44.131917 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:44.132453 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:44.132936 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:23:44.132939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:23:44.132942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:23:44.133190 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:23:44.133891 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:23:44.133903 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:23:44.134298 (+ 395us) client_negotiation.cc:769] Sending connection context
0925 09:23:44.134598 (+ 300us) client_negotiation.cc:240] Negotiation successful
0925 09:23:44.135035 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":390,"threads_started":1}
I20260925 09:23:44.137741 19414 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37625
I20260925 09:23:44.138092 19414 heartbeater.cc:464] Registering TS with master...
I20260925 09:23:44.138851 19414 heartbeater.cc:511] Master 127.3.82.62:37625 requested a full tablet report, sending...
I20260925 09:23:44.140646 18957 ts_manager.cc:194] Registered new tserver with Master: 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3:46313)
I20260925 09:23:44.141604 18957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:46647
I20260925 09:23:44.142169 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:23:44.160858 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.147032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41526 (local address 127.3.82.62:37625)
0925 09:23:44.147247 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.147253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.147644 (+ 391us) server_negotiation.cc:407] Connection header received
0925 09:23:44.147808 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.147812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.147883 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.147983 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:44.149421 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.150346 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.151852 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.152329 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.152473 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.155380 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:44.155385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:44.155389 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:44.155428 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:44.158299 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.159035 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.159041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.159044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.159170 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.159549 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.159552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.159555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.159893 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:44.160074 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.160391 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.160614 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 09:23:44.165794 18957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:23:44.173368 18957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:23:44.209735 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.187168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:46313 (local address 127.0.0.1:54596)
0925 09:23:44.198595 (+ 11427us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:44.198641 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:44.198869 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:44.201478 (+ 2609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:44.201486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:44.201530 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:44.202028 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.202042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.204972 (+ 2930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.204980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.207091 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.207106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.208450 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.208457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.208767 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.208878 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 09:23:44.209029 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 09:23:44.209236 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11120,"mutex_wait_us":54,"thread_start_us":14840,"threads_started":1}
I20260925 09:23:44.210559 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.187545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.3.82.3:46313)
0925 09:23:44.200875 (+ 13330us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.200882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.200910 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:23:44.201002 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.201008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.201218 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.201454 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:44.202260 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.204770 (+ 2510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.207265 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.208237 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.208402 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.208655 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.210042 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.210223 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13167,"thread_start_us":14161,"threads_started":1}
I20260925 09:23:44.212030 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.187476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42383 (local address 127.0.0.1:36556)
0925 09:23:44.201708 (+ 14232us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:44.201737 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:44.201906 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:44.203741 (+ 1835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:44.203745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:44.203769 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:44.204182 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.204195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.206233 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.206243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.208857 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.208872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.211132 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.211138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.211332 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.211364 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:23:44.211457 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 09:23:44.211591 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14025,"mutex_wait_us":3,"thread_start_us":10649,"threads_started":1}
I20260925 09:23:44.212131 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.188011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:43023 (local address 127.0.0.1:58424)
0925 09:23:44.203408 (+ 15397us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:44.203423 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:44.203574 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:44.204410 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:44.204414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:44.204444 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:44.204829 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.204841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.206735 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.206740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.208949 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.208963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.211267 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.211273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.211455 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.211503 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 09:23:44.211624 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 09:23:44.211759 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15298,"thread_start_us":15051,"threads_started":1}
I20260925 09:23:44.212606 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.189012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.3.82.1:43023)
0925 09:23:44.202873 (+ 13861us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.202884 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.203533 (+ 649us) server_negotiation.cc:407] Connection header received
0925 09:23:44.203739 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.203747 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.203976 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.204175 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:44.205042 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.206575 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.209113 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.211046 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.211202 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.211430 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.211671 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.211847 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13689,"thread_start_us":13524,"threads_started":1}
I20260925 09:23:44.212968 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.188424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.3.82.2:42383)
0925 09:23:44.202955 (+ 14531us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.202967 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.203005 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:23:44.203111 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.203119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.203386 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.203591 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:44.204371 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.206075 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.209056 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.210634 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.210793 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.210998 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.212443 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.212622 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14372,"thread_start_us":14313,"threads_started":1}
I20260925 09:23:44.215024 19349 tablet_service.cc:1467] Processing CreateTablet for tablet 2b7190fb19cd41f9929620918d3a78e2 (DEFAULT_TABLE table=test-table [id=8d8d89b9de584d57a792f0248408324b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:44.216111 19081 tablet_service.cc:1467] Processing CreateTablet for tablet 2b7190fb19cd41f9929620918d3a78e2 (DEFAULT_TABLE table=test-table [id=8d8d89b9de584d57a792f0248408324b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:44.216671 19349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b7190fb19cd41f9929620918d3a78e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:44.216459 19215 tablet_service.cc:1467] Processing CreateTablet for tablet 2b7190fb19cd41f9929620918d3a78e2 (DEFAULT_TABLE table=test-table [id=8d8d89b9de584d57a792f0248408324b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:23:44.217394 19081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b7190fb19cd41f9929620918d3a78e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:44.217758 19215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b7190fb19cd41f9929620918d3a78e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:23:44.229480 19433 tablet_bootstrap.cc:492] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Bootstrap starting.
I20260925 09:23:44.231941 19435 tablet_bootstrap.cc:492] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Bootstrap starting.
I20260925 09:23:44.234565 19434 tablet_bootstrap.cc:492] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Bootstrap starting.
I20260925 09:23:44.234627 19433 tablet_bootstrap.cc:654] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:44.235880 19433 log.cc:824] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:44.235975 19435 tablet_bootstrap.cc:654] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:44.237273 19435 log.cc:824] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:44.238435 19434 tablet_bootstrap.cc:654] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Neither blocks nor log segments found. Creating new log.
I20260925 09:23:44.239234 19433 tablet_bootstrap.cc:492] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: No bootstrap required, opened a new log
I20260925 09:23:44.239521 19433 ts_tablet_manager.cc:1397] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260925 09:23:44.239867 19434 log.cc:824] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Log is configured to *not* fsync() on all Append() calls
I20260925 09:23:44.240634 19435 tablet_bootstrap.cc:492] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: No bootstrap required, opened a new log
I20260925 09:23:44.240931 19435 ts_tablet_manager.cc:1397] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260925 09:23:44.243709 19434 tablet_bootstrap.cc:492] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: No bootstrap required, opened a new log
I20260925 09:23:44.244022 19434 ts_tablet_manager.cc:1397] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260925 09:23:44.248684 19433 raft_consensus.cc:348] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.249138 19433 raft_consensus.cc:374] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:44.249259 19433 raft_consensus.cc:729] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a5649eddd74c30a98ffb2b3e275bb3, State: Initialized, Role: FOLLOWER
I20260925 09:23:44.250021 19433 consensus_queue.cc:260] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.252020 19435 raft_consensus.cc:348] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.252496 19435 raft_consensus.cc:374] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:44.252588 19435 raft_consensus.cc:729] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e16ceb5871241e5a95f693949484528, State: Initialized, Role: FOLLOWER
I20260925 09:23:44.253295 19433 ts_tablet_manager.cc:1428] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260925 09:23:44.253340 19435 consensus_queue.cc:260] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.255515 19434 raft_consensus.cc:348] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.255918 19434 raft_consensus.cc:374] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:23:44.256028 19434 raft_consensus.cc:729] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0b6054639e4518bf894fa7ae3a61f4, State: Initialized, Role: FOLLOWER
I20260925 09:23:44.256693 19434 consensus_queue.cc:260] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.257067 19280 heartbeater.cc:503] Master 127.3.82.62:37625 was elected leader, sending a full tablet report...
I20260925 09:23:44.257851 19414 heartbeater.cc:503] Master 127.3.82.62:37625 was elected leader, sending a full tablet report...
I20260925 09:23:44.258658 19434 ts_tablet_manager.cc:1428] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260925 09:23:44.259879 19435 ts_tablet_manager.cc:1428] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Time spent starting tablet: real 0.019s user 0.008s sys 0.010s
W20260925 09:23:44.300252 19281 tablet.cc:2369] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:44.336381 19441 raft_consensus.cc:482] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:23:44.336674 19441 raft_consensus.cc:504] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.338557 19441 leader_election.cc:288] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e16ceb5871241e5a95f693949484528 (127.3.82.2:42383), 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023)
I20260925 09:23:44.346268 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.339034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:58953 (local address 127.3.82.1:43023)
0925 09:23:44.339256 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.339262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.339288 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:23:44.339483 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.339492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.339593 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.339700 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:44.340176 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.341214 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.343762 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.345157 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.345330 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.345462 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.345796 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.345981 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:23:44.346295 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.338915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:43023 (local address 127.3.82.3:58953)
0925 09:23:44.339122 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:44.339145 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:44.339300 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:44.339746 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:44.339752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:44.339785 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:44.340005 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.340020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.341376 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.341380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.343518 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.343535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.345357 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.345362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.345584 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.345632 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 09:23:44.345723 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 09:23:44.345888 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 09:23:44.346851 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.339050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:38089 (local address 127.3.82.2:42383)
0925 09:23:44.339267 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.339273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.340046 (+ 773us) server_negotiation.cc:407] Connection header received
0925 09:23:44.340266 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.340274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.340347 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.340446 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:23:44.341107 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.341918 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.344257 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.345752 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.345932 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.346037 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.346411 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.346576 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 09:23:44.347434 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.338934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42383 (local address 127.3.82.3:38089)
0925 09:23:44.339924 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0925 09:23:44.339950 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:23:44.340070 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:23:44.340501 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:23:44.340505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:23:44.340561 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:23:44.340948 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.340959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.342057 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.342062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.344081 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:23:44.344092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.345995 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.346000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:23:44.346187 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.346218 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:23:44.347045 (+ 827us) client_negotiation.cc:240] Negotiation successful
0925 09:23:44.347172 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":145,"thread_start_us":460,"threads_started":1}
I20260925 09:23:44.347502 19101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7190fb19cd41f9929620918d3a78e2" candidate_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" is_pre_election: true
I20260925 09:23:44.348196 19101 raft_consensus.cc:2454] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a0b6054639e4518bf894fa7ae3a61f4 in term 0.
I20260925 09:23:44.348430 19235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7190fb19cd41f9929620918d3a78e2" candidate_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e16ceb5871241e5a95f693949484528" is_pre_election: true
I20260925 09:23:44.349004 19302 leader_election.cc:302] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0b6054639e4518bf894fa7ae3a61f4, 86a5649eddd74c30a98ffb2b3e275bb3; no voters:
I20260925 09:23:44.348994 19235 raft_consensus.cc:2454] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a0b6054639e4518bf894fa7ae3a61f4 in term 0.
I20260925 09:23:44.349458 19441 raft_consensus.cc:2781] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:23:44.349659 19441 raft_consensus.cc:482] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:23:44.349783 19441 raft_consensus.cc:3037] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:44.352250 19441 raft_consensus.cc:504] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:23:44.352962 19441 leader_election.cc:288] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [CANDIDATE]: Term 1 election: Requested vote from peers 0e16ceb5871241e5a95f693949484528 (127.3.82.2:42383), 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023)
I20260925 09:23:44.353505 19235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7190fb19cd41f9929620918d3a78e2" candidate_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e16ceb5871241e5a95f693949484528"
I20260925 09:23:44.353750 19235 raft_consensus.cc:3037] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:44.353717 19101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7190fb19cd41f9929620918d3a78e2" candidate_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
I20260925 09:23:44.354007 19101 raft_consensus.cc:3037] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:23:44.356143 19235 raft_consensus.cc:2454] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a0b6054639e4518bf894fa7ae3a61f4 in term 1.
I20260925 09:23:44.356292 19101 raft_consensus.cc:2454] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a0b6054639e4518bf894fa7ae3a61f4 in term 1.
I20260925 09:23:44.356711 19305 leader_election.cc:302] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e16ceb5871241e5a95f693949484528, 1a0b6054639e4518bf894fa7ae3a61f4; no voters:
I20260925 09:23:44.357128 19441 raft_consensus.cc:2781] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:23:44.357613 19441 raft_consensus.cc:686] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 LEADER]: Becoming Leader. State: Replica: 1a0b6054639e4518bf894fa7ae3a61f4, State: Running, Role: LEADER
I20260925 09:23:44.358232 19441 consensus_queue.cc:237] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
W20260925 09:23:44.363557 19415 tablet.cc:2369] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:23:44.363641 18956 catalog_manager.cc:5495] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } health_report { overall_health: UNKNOWN } } }
I20260925 09:23:44.411473 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.405087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54612 (local address 127.3.82.3:46313)
0925 09:23:44.405880 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.405886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.405905 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:23:44.406029 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.406035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.406140 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.406301 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:23:44.406710 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.407484 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.409014 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.409423 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.409556 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.410125 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:23:44.410650 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:23:44.410825 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.410905 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.410991 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602}
I20260925 09:23:44.425374 19101 raft_consensus.cc:1261] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Refusing update from remote peer 1a0b6054639e4518bf894fa7ae3a61f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:44.425603 19235 raft_consensus.cc:1261] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Refusing update from remote peer 1a0b6054639e4518bf894fa7ae3a61f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:23:44.426687 19441 consensus_queue.cc:1035] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:44.427436 19441 consensus_queue.cc:1035] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:23:44.447032 19446 mvcc.cc:205] Tried to move back new op lower bound from 7333184407230816256 to 7333184406976454656. Current Snapshot: MvccSnapshot[applied={T|T < 7333184407230816256}]
I20260925 09:23:44.448251 19447 mvcc.cc:205] Tried to move back new op lower bound from 7333184407230816256 to 7333184406976454656. Current Snapshot: MvccSnapshot[applied={T|T < 7333184407230816256}]
I20260925 09:23:44.455161 19448 mvcc.cc:205] Tried to move back new op lower bound from 7333184407230816256 to 7333184406976454656. Current Snapshot: MvccSnapshot[applied={T|T < 7333184407230816256}]
I20260925 09:23:44.466763 18957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:44.467116 18957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:23:44.472565 18957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Sending DeleteTablet for 3 replicas of tablet 2b7190fb19cd41f9929620918d3a78e2
I20260925 09:23:44.474258 19349 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b7190fb19cd41f9929620918d3a78e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:44 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:54596
I20260925 09:23:44.474596 19081 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b7190fb19cd41f9929620918d3a78e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:44 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:58424
I20260925 09:23:44.474854 19215 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b7190fb19cd41f9929620918d3a78e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:23:44 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:36556
I20260925 09:23:44.475951 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19018
I20260925 09:23:44.504175 19455 tablet_replica.cc:330] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: stopping tablet replica
I20260925 09:23:44.504896 19455 raft_consensus.cc:2229] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:23:44.510681 19455 raft_consensus.cc:2258] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:44.511446 19456 tablet_replica.cc:330] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: stopping tablet replica
I20260925 09:23:44.512066 19456 raft_consensus.cc:2229] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:44.513023 19456 raft_consensus.cc:2258] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:44.515177 19455 ts_tablet_manager.cc:1905] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:44.519505 19456 ts_tablet_manager.cc:1905] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:44.523973 19457 tablet_replica.cc:330] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: stopping tablet replica
I20260925 09:23:44.524694 19457 raft_consensus.cc:2229] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:23:44.525441 19457 raft_consensus.cc:2258] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:23:44.525784 19456 ts_tablet_manager.cc:1918] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:44.525990 19456 log.cc:1196] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/2b7190fb19cd41f9929620918d3a78e2
I20260925 09:23:44.526818 19456 ts_tablet_manager.cc:1939] T 2b7190fb19cd41f9929620918d3a78e2 P 86a5649eddd74c30a98ffb2b3e275bb3: Deleting consensus metadata
I20260925 09:23:44.528956 19457 ts_tablet_manager.cc:1905] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:23:44.528932 18941 catalog_manager.cc:4841] TS 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023): tablet 2b7190fb19cd41f9929620918d3a78e2 (table test-table [id=8d8d89b9de584d57a792f0248408324b]) successfully deleted
Sep 25 09:23:44 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:23:44.537473 19455 ts_tablet_manager.cc:1918] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:44.537715 19457 ts_tablet_manager.cc:1918] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:23:44.537940 19457 log.cc:1196] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/2b7190fb19cd41f9929620918d3a78e2
I20260925 09:23:44.538574 19457 ts_tablet_manager.cc:1939] T 2b7190fb19cd41f9929620918d3a78e2 P 0e16ceb5871241e5a95f693949484528: Deleting consensus metadata
I20260925 09:23:44.540855 18944 catalog_manager.cc:4841] TS 0e16ceb5871241e5a95f693949484528 (127.3.82.2:42383): tablet 2b7190fb19cd41f9929620918d3a78e2 (table test-table [id=8d8d89b9de584d57a792f0248408324b]) successfully deleted
I20260925 09:23:44.547484 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.477579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.3.82.1:43023)
0925 09:23:44.477862 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.477873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.500211 (+ 22338us) server_negotiation.cc:407] Connection header received
0925 09:23:44.500857 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.500865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.500995 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.501084 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:44.505355 (+ 4271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.506932 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.515449 (+ 8517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.516283 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.516424 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.537996 (+ 21572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:44.538030 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:44.538052 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:44.538119 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:44.542340 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.543661 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.543673 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.543677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.543856 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.544393 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.544400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.544404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.544986 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:44.545324 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.546765 (+ 1441us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.547074 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 09:23:44.540226 19455 log.cc:1196] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/2b7190fb19cd41f9929620918d3a78e2
I20260925 09:23:44.550611 19455 ts_tablet_manager.cc:1939] T 2b7190fb19cd41f9929620918d3a78e2 P 1a0b6054639e4518bf894fa7ae3a61f4: Deleting consensus metadata
I20260925 09:23:44.550781 19141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:44.552558 18942 catalog_manager.cc:4841] TS 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3:46313): tablet 2b7190fb19cd41f9929620918d3a78e2 (table test-table [id=8d8d89b9de584d57a792f0248408324b]) successfully deleted
I20260925 09:23:44.725659 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19018
I20260925 09:23:44.755103 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19152
Sep 25 09:23:44 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:23:44.776232 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.756551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.3.82.2:42383)
0925 09:23:44.756776 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.756780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.756798 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:23:44.756853 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.756857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.756935 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.757024 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:44.759082 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.759915 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.762170 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.762587 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.762695 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.768648 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:44.768666 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:44.768681 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:44.768747 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:44.772820 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.773928 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.773937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.773940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.774099 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.774547 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.774552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.774560 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.774961 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:44.775286 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.775621 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.775870 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260925 09:23:44.776926 19275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:44.869792 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19152
I20260925 09:23:44.887705 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19286
Sep 25 09:23:44 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:23:44.907790 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:23:44.888557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.3.82.3:46313)
0925 09:23:44.888769 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 09:23:44.888777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:23:44.888808 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:23:44.888923 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:23:44.888928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:23:44.889044 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:23:44.889181 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:23:44.890697 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.892031 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.893440 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:23:44.894218 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:23:44.894362 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:23:44.899374 (+ 5012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:23:44.899396 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:23:44.899419 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:23:44.899496 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:23:44.903910 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.904993 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.905003 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.905005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.905195 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:23:44.905665 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:23:44.905671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:23:44.905675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:23:44.906339 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:23:44.906713 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0925 09:23:44.907078 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 09:23:44.907342 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 09:23:44.908708 19409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:45.040787 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19286
I20260925 09:23:45.061296 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 18925
I20260925 09:23:45.062642 18987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:23:45.181252 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 18925
I20260925 09:24:05.203822 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37625
--webserver_interface=127.3.82.62
--webserver_port=40223
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:05.555269 19463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:05.555598 19463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:05.555688 19463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:05.563210 19463 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 09:24:05.563328 19463 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 09:24:05.563373 19463 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 09:24:05.563419 19463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:24:05.563475 19463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:05.563544 19463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:05.563589 19463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:24:05.563637 19463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:24:05.574100 19463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=40223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:05.575400 19463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:05.576691 19463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:05.602007 19470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:05.601999 19472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:05.602120 19469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:05.602895 19463 server_base.cc:1034] running on GCE node
I20260925 09:24:05.603994 19463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:05.605983 19463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:05.607358 19463 hybrid_clock.cc:648] HybridClock initialized: now 1790328245607276 us; error 69 us; skew 500 ppm
I20260925 09:24:05.607918 19463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:05 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328245, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:05.612510 19463 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:24:05.614619 19463 webserver.cc:460] Webserver started at http://127.3.82.62:40223/ using document root <none> and password file <none>
I20260925 09:24:05.615437 19463 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:05.615553 19463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:05.622182 19463 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 09:24:05.625777 19479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:05.627640 19463 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:24:05.627874 19463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "bc335470a63f48318196a52ffed6341d"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "e4686f6c4fb9334c70f49ed5986ccd67"
server_key_iv: "33cee414889439be871f2a68ab0ee2a1"
server_key_version: "encryptionkey@0"
I20260925 09:24:05.629040 19463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:05.651899 19463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:05.657141 19463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:05.657416 19463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:05.677670 19463 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37625
I20260925 09:24:05.677695 19530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37625 every 8 connection(s)
I20260925 09:24:05.679366 19463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:24:05.681535 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19463
I20260925 09:24:05.682523 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:43023
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:37625
--webserver_port=37329
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:24:05.687235 19531 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:24:05.690991 19531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap starting.
I20260925 09:24:05.712479 19531 log.cc:824] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:05.726300 19531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:24:05.726981 19531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap complete.
I20260925 09:24:05.737984 19531 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:05.739035 19531 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc335470a63f48318196a52ffed6341d, State: Initialized, Role: FOLLOWER
I20260925 09:24:05.739794 19531 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:05.740087 19531 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:24:05.740247 19531 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:24:05.740453 19531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:24:05.745239 19531 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:05.745934 19531 leader_election.cc:302] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc335470a63f48318196a52ffed6341d; no voters:
I20260925 09:24:05.746680 19531 leader_election.cc:288] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 09:24:05.746847 19535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:24:05.748071 19535 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 LEADER]: Becoming Leader. State: Replica: bc335470a63f48318196a52ffed6341d, State: Running, Role: LEADER
I20260925 09:24:05.748756 19535 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:05.749336 19531 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:24:05.751636 19537 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc335470a63f48318196a52ffed6341d. Latest consensus state: current_term: 2 leader_uuid: "bc335470a63f48318196a52ffed6341d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } } }
I20260925 09:24:05.752050 19537 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:05.751736 19536 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bc335470a63f48318196a52ffed6341d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } } }
I20260925 09:24:05.752220 19536 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:05.763862 19541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:24:05.774724 19541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:24:05.776492 19541 catalog_manager.cc:1259] Loaded cluster ID: 10960b0e7b5349dbb028f475efaa6180
I20260925 09:24:05.776621 19541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:24:05.780814 19541 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:24:05.782491 19541 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Loaded TSK: 0
I20260925 09:24:05.823334 19541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Generated new TSK 1
I20260925 09:24:05.823796 19541 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:24:06.082644 19533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:06.083062 19533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:06.083321 19533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:06.092254 19533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:06.092445 19533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:06.092659 19533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:24:06.104661 19533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:43023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=37329
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:06.106272 19533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:06.107843 19533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:06.124594 19559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:06.125041 19561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:06.125499 19533 server_base.cc:1034] running on GCE node
W20260925 09:24:06.125921 19558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:06.126654 19533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:06.127827 19533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:06.129125 19533 hybrid_clock.cc:648] HybridClock initialized: now 1790328246129052 us; error 65 us; skew 500 ppm
I20260925 09:24:06.129525 19533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:06 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328246, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:06.133440 19533 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:24:06.135588 19533 webserver.cc:460] Webserver started at http://127.3.82.1:37329/ using document root <none> and password file <none>
I20260925 09:24:06.136621 19533 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:06.136803 19533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:06.145637 19533 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:24:06.150323 19568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:06.152740 19533 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:24:06.152979 19533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "aaac84088bd49f5d9bfcf9299bf2dadb"
server_key_iv: "08fc64156c4b716a03cab335e595d19d"
server_key_version: "encryptionkey@0"
I20260925 09:24:06.154727 19533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:06.182989 19533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:06.189893 19533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:06.190250 19533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:06.191752 19533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:06.193696 19533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:06.193818 19533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:06.193960 19533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:06.194041 19533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:06.241030 19533 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:43023
I20260925 09:24:06.241080 19680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:43023 every 8 connection(s)
I20260925 09:24:06.243147 19533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:24:06.244329 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19533
I20260925 09:24:06.245180 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:42383
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:37625
--webserver_port=38907
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:24:06 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328246, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:06.300072 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:06.247252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:50023)
0925 09:24:06.271994 (+ 24742us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:06.272064 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:06.275275 (+ 3211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:06.278051 (+ 2776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:06.278066 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:06.278995 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:06.279905 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:06.279938 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.282561 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.282566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:06.284075 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:06.284093 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.285996 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.286004 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:06.286246 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:06.287102 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:06.287137 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:06.290031 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:06.295184 (+ 5153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:06.295192 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:06.295206 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:06.295761 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:06.296377 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:06.296387 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:06.296389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:06.296619 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:06.297590 (+ 971us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:06.297597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:06.298076 (+ 479us) client_negotiation.cc:769] Sending connection context
0925 09:24:06.298377 (+ 301us) client_negotiation.cc:240] Negotiation successful
0925 09:24:06.298908 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24365,"thread_start_us":24113,"threads_started":1}
I20260925 09:24:06.300420 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:06.248274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:50023 (local address 127.3.82.62:37625)
0925 09:24:06.270884 (+ 22610us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:06.270915 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:06.272622 (+ 1707us) server_negotiation.cc:407] Connection header received
0925 09:24:06.276064 (+ 3442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:06.276091 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:06.276637 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:06.277065 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:06.280198 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.282320 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.284291 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.285599 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.285887 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:06.290406 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:06.290421 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:06.290450 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:06.290520 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:06.294971 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:06.295939 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:06.295948 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:06.295956 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:06.296200 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:06.296754 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:06.296760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:06.296764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:06.297420 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:06.297622 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:06.298646 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0925 09:24:06.299124 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22206,"thread_start_us":22666,"threads_started":1}
I20260925 09:24:06.303435 19681 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37625
I20260925 09:24:06.303807 19681 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:06.304872 19681 heartbeater.cc:511] Master 127.3.82.62:37625 requested a full tablet report, sending...
I20260925 09:24:06.307420 19496 ts_manager.cc:194] Registered new tserver with Master: 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023)
I20260925 09:24:06.308966 19496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:50023
W20260925 09:24:06.625735 19685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:06.626080 19685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:06.626293 19685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:06.634560 19685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:06.634686 19685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:06.634898 19685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:24:06.645897 19685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:42383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=38907
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:06.647318 19685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:06.648733 19685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:06.665220 19690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:06.665189 19693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:06.665272 19691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:06.665964 19685 server_base.cc:1034] running on GCE node
I20260925 09:24:06.666800 19685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:06.667917 19685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:06.669178 19685 hybrid_clock.cc:648] HybridClock initialized: now 1790328246669102 us; error 72 us; skew 500 ppm
I20260925 09:24:06.669535 19685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:06 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328246, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:06.673069 19685 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:24:06.675107 19685 webserver.cc:460] Webserver started at http://127.3.82.2:38907/ using document root <none> and password file <none>
I20260925 09:24:06.675868 19685 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:06.676034 19685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:06.684024 19685 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:24:06.688078 19700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:06.690246 19685 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:24:06.690454 19685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "0e16ceb5871241e5a95f693949484528"
format_stamp: "Formatted at 2026-09-25 09:23:43 on dist-test-slave-4wm5"
server_key: "02bdf4387ce49541578cddb73fe66451"
server_key_iv: "0bbbe861d6df931c48942f9c013f7ec6"
server_key_version: "encryptionkey@0"
I20260925 09:24:06.691874 19685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:06.714609 19685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:06.719491 19685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:06.719740 19685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:06.720777 19685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:06.722213 19685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:06.722333 19685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:06.722425 19685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:06.722481 19685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:06.761008 19685 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:42383
I20260925 09:24:06.761135 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:42383 every 8 connection(s)
I20260925 09:24:06.762916 19685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:24:06.764626 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19685
I20260925 09:24:06.765534 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:46313
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:37625
--webserver_port=46765
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 09:24:06 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328246, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:06.793593 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:06.765907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:41413 (local address 127.3.82.62:37625)
0925 09:24:06.766205 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:06.766213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:06.772863 (+ 6650us) server_negotiation.cc:407] Connection header received
0925 09:24:06.775143 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:06.775149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:06.775325 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:06.775450 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:06.777256 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.778531 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.780307 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.781122 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.781309 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:06.785147 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:06.785158 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:06.785166 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:06.785230 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:06.789588 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:06.790594 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:06.790602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:06.790609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:06.790794 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:06.791273 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:06.791282 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:06.791287 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:06.791736 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:06.791915 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:06.792669 (+ 754us) server_negotiation.cc:299] Negotiation successful
0925 09:24:06.792952 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 09:24:06.794155 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:06.766505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:41413)
0925 09:24:06.772594 (+ 6089us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:06.772656 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:06.774749 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:06.775751 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:06.775761 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:06.776356 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:06.777006 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:06.777025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.778743 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.778747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:06.780115 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:06.780129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:06.781387 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:06.781392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:06.781627 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:06.782430 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:06.782468 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:06.784820 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:06.789873 (+ 5053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:06.789890 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:06.789906 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:06.790418 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:06.790927 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:06.790930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:06.790934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:06.791128 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:06.791869 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:06.791875 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:06.792248 (+ 373us) client_negotiation.cc:769] Sending connection context
0925 09:24:06.792500 (+ 252us) client_negotiation.cc:240] Negotiation successful
0925 09:24:06.792930 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5764,"thread_start_us":5553,"threads_started":1}
I20260925 09:24:06.795821 19813 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37625
I20260925 09:24:06.796144 19813 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:06.797065 19813 heartbeater.cc:511] Master 127.3.82.62:37625 requested a full tablet report, sending...
I20260925 09:24:06.798887 19496 ts_manager.cc:194] Registered new tserver with Master: 0e16ceb5871241e5a95f693949484528 (127.3.82.2:42383)
I20260925 09:24:06.799995 19496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:41413
W20260925 09:24:07.126914 19816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:07.127224 19816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:07.127324 19816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:07.135277 19816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:07.135401 19816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:07.135519 19816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:24:07.149073 19816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:46313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=46765
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:07.150980 19816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:07.152662 19816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:07.172091 19821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:07.172765 19824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:07.172794 19822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:07.173699 19816 server_base.cc:1034] running on GCE node
I20260925 09:24:07.174531 19816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:07.175609 19816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:07.176904 19816 hybrid_clock.cc:648] HybridClock initialized: now 1790328247176822 us; error 75 us; skew 500 ppm
I20260925 09:24:07.177279 19816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:07 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328247, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:07.181154 19816 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:24:07.182956 19816 webserver.cc:460] Webserver started at http://127.3.82.3:46765/ using document root <none> and password file <none>
I20260925 09:24:07.183678 19816 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:07.183802 19816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:07.189795 19816 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:24:07.192991 19831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:07.194624 19816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:24:07.194784 19816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "1a0b6054639e4518bf894fa7ae3a61f4"
format_stamp: "Formatted at 2026-09-25 09:23:44 on dist-test-slave-4wm5"
server_key: "2d85f87becf97986e10d42140917aaec"
server_key_iv: "b3d70366660708a558b91ace2b3b64de"
server_key_version: "encryptionkey@0"
I20260925 09:24:07.195793 19816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:07.234776 19816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:07.239559 19816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:07.239794 19816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:07.241117 19816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:07.242969 19816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:07.243103 19816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:07.243211 19816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:07.243292 19816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:07.282241 19816 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:46313
I20260925 09:24:07.282341 19943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:46313 every 8 connection(s)
I20260925 09:24:07.283788 19816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:24:07.286494 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19816
Sep 25 09:24:07 dist-test-slave-4wm5 krb5kdc[18912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328247, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:07.304337 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.287589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.3.82.62:37625)
0925 09:24:07.288131 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.288136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.288617 (+ 481us) server_negotiation.cc:407] Connection header received
0925 09:24:07.288851 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.288858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.288956 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.289096 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:07.291054 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.292204 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.293647 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.294269 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.294435 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.297913 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:07.297919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:07.297922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:07.297958 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:07.301060 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.301983 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.301990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.301993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.302191 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.302646 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.302650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.302652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.303004 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:07.303333 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.303663 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.303914 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":343,"threads_started":1}
I20260925 09:24:07.309341 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.286810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:54215 (local address 127.3.82.62:37625)
0925 09:24:07.287047 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.287054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.289611 (+ 2557us) server_negotiation.cc:407] Connection header received
0925 09:24:07.291926 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.291934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.292001 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.292087 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:07.293792 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.294739 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.297401 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.297947 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.298162 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.302167 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:07.302172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:07.302176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:07.302208 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:07.305421 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.306584 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.306588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.306590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.306724 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.307300 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.307304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.307306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.307697 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:07.307859 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.308715 (+ 856us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.308960 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:24:07.310504 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.287413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:54215)
0925 09:24:07.289443 (+ 2030us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:07.289483 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:07.291588 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:07.292414 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:07.292423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:07.292967 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:07.293557 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.293611 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.294937 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.294943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.297190 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.297210 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.298311 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.298318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.298569 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.299319 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:07.299349 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:07.301916 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:07.305645 (+ 3729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:07.305657 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:07.305677 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:07.306435 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:07.306852 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:07.306856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:07.306860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:07.307141 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:07.307838 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:07.307846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:07.308330 (+ 484us) client_negotiation.cc:769] Sending connection context
0925 09:24:07.308614 (+ 284us) client_negotiation.cc:240] Negotiation successful
0925 09:24:07.309215 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1661,"thread_start_us":1390,"threads_started":1}
I20260925 09:24:07.312184 19944 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37625
I20260925 09:24:07.312603 19944 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:07.312582 19681 heartbeater.cc:503] Master 127.3.82.62:37625 was elected leader, sending a full tablet report...
I20260925 09:24:07.313534 19944 heartbeater.cc:511] Master 127.3.82.62:37625 requested a full tablet report, sending...
I20260925 09:24:07.315414 19496 ts_manager.cc:194] Registered new tserver with Master: 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3:46313)
I20260925 09:24:07.316321 19496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:54215
I20260925 09:24:07.320305 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:24:07.327984 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.321424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.3.82.62:37625)
0925 09:24:07.321610 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.321615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.322162 (+ 547us) server_negotiation.cc:407] Connection header received
0925 09:24:07.322411 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.322415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.322566 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.322699 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:24:07.323125 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.323986 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.325787 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.326438 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.326548 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.327195 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:24:07.327438 (+ 243us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 09:24:07.327775 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60308: token expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260925 09:24:07.328150 19684 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60308: token expired
W20260925 09:24:07.328827 19418 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.82.62:37625)
I20260925 09:24:07.343925 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.329445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.3.82.62:37625)
0925 09:24:07.329633 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.329637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.329738 (+ 101us) server_negotiation.cc:407] Connection header received
0925 09:24:07.329987 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.329991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.330078 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.330174 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:07.331956 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.332800 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.334722 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.335165 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.335273 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.338358 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:07.338363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:07.338367 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:07.338406 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:07.341048 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.341770 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.341775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.341777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.342025 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.342430 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.342434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.342436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.342791 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:07.343011 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.343387 (+ 376us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.343583 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 09:24:07.348114 19496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:24:07.356101 19496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:24:07.391795 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.369960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42383 (local address 127.0.0.1:51238)
0925 09:24:07.381945 (+ 11985us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:07.381969 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:07.382108 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:07.384108 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:07.384115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:07.384153 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:07.384668 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.384681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.386356 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.386361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.388377 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.388394 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.390808 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.390813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.391038 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.391090 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 09:24:07.391223 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 09:24:07.391415 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11784,"thread_start_us":10700,"threads_started":1}
I20260925 09:24:07.392616 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.370129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.3.82.2:42383)
0925 09:24:07.383470 (+ 13341us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.383479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.383507 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:24:07.383596 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.383603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.383830 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.384084 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:07.384872 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.386187 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.388588 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.390316 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.390443 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.390652 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.392126 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.392314 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13180,"thread_start_us":14160,"threads_started":1}
I20260925 09:24:07.397322 19956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.369429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:46313 (local address 127.0.0.1:41450)
0925 09:24:07.381107 (+ 11678us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:07.381156 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:07.381374 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:07.389730 (+ 8356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:07.389735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:07.389762 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:07.390199 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.390212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.392093 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.392098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.394293 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.394306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.396181 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.396187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.396493 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.396524 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 09:24:07.396597 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 09:24:07.396756 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11371,"mutex_wait_us":78,"thread_start_us":12147,"threads_started":1}
I20260925 09:24:07.397357 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.369775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.3.82.3:46313)
0925 09:24:07.389129 (+ 19354us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.389135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.389162 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:24:07.389241 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.389246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.389487 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.390430 (+ 943us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:07.390756 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.391934 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.394517 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.395886 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.396461 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.396693 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.396775 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.396946 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19176,"thread_start_us":18916,"threads_started":1}
I20260925 09:24:07.396672 19748 tablet_service.cc:1467] Processing CreateTablet for tablet 1c979ad80c3045d08dcfb27f16479e95 (DEFAULT_TABLE table=test-table [id=b0550c53e9e348f4a9928e8abb8ec002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:07.397789 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.370204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:43023 (local address 127.0.0.1:32910)
0925 09:24:07.388030 (+ 17826us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:07.388056 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:07.388232 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:07.389247 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:07.389253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:07.389307 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:07.389668 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.389682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.391636 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.391642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.394030 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.394047 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.396146 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.396156 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.396386 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.396447 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 09:24:07.396542 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 09:24:07.396757 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17606,"mutex_wait_us":29,"thread_start_us":18177,"threads_started":1}
I20260925 09:24:07.398207 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.370823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.3.82.1:43023)
0925 09:24:07.379421 (+ 8598us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.379435 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.388220 (+ 8785us) server_negotiation.cc:407] Connection header received
0925 09:24:07.388433 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.388445 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.388758 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.388980 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:07.389897 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.391466 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.394251 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.395851 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.396012 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.396275 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.397365 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.397574 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8314,"thread_start_us":8103,"threads_started":1}
I20260925 09:24:07.398733 19748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c979ad80c3045d08dcfb27f16479e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:07.401875 19879 tablet_service.cc:1467] Processing CreateTablet for tablet 1c979ad80c3045d08dcfb27f16479e95 (DEFAULT_TABLE table=test-table [id=b0550c53e9e348f4a9928e8abb8ec002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:07.401773 19616 tablet_service.cc:1467] Processing CreateTablet for tablet 1c979ad80c3045d08dcfb27f16479e95 (DEFAULT_TABLE table=test-table [id=b0550c53e9e348f4a9928e8abb8ec002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:07.403105 19616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c979ad80c3045d08dcfb27f16479e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:07.403180 19879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c979ad80c3045d08dcfb27f16479e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:07.416491 19960 tablet_bootstrap.cc:492] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Bootstrap starting.
I20260925 09:24:07.418916 19961 tablet_bootstrap.cc:492] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Bootstrap starting.
I20260925 09:24:07.420584 19960 tablet_bootstrap.cc:654] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:07.422595 19962 tablet_bootstrap.cc:492] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Bootstrap starting.
I20260925 09:24:07.422578 19960 log.cc:824] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:07.423820 19961 tablet_bootstrap.cc:654] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:07.425071 19961 log.cc:824] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:07.426177 19962 tablet_bootstrap.cc:654] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:07.426337 19960 tablet_bootstrap.cc:492] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: No bootstrap required, opened a new log
I20260925 09:24:07.426685 19960 ts_tablet_manager.cc:1397] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260925 09:24:07.427318 19962 log.cc:824] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:07.428467 19961 tablet_bootstrap.cc:492] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: No bootstrap required, opened a new log
I20260925 09:24:07.428778 19961 ts_tablet_manager.cc:1397] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260925 09:24:07.430011 19962 tablet_bootstrap.cc:492] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: No bootstrap required, opened a new log
I20260925 09:24:07.430264 19962 ts_tablet_manager.cc:1397] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260925 09:24:07.437795 19960 raft_consensus.cc:348] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.438397 19960 raft_consensus.cc:374] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:07.438509 19960 raft_consensus.cc:729] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0b6054639e4518bf894fa7ae3a61f4, State: Initialized, Role: FOLLOWER
I20260925 09:24:07.438284 19962 raft_consensus.cc:348] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.438802 19962 raft_consensus.cc:374] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:07.438936 19962 raft_consensus.cc:729] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e16ceb5871241e5a95f693949484528, State: Initialized, Role: FOLLOWER
I20260925 09:24:07.439239 19960 consensus_queue.cc:260] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.439572 19962 consensus_queue.cc:260] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.440083 19961 raft_consensus.cc:348] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.440548 19961 raft_consensus.cc:374] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:07.440639 19961 raft_consensus.cc:729] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a5649eddd74c30a98ffb2b3e275bb3, State: Initialized, Role: FOLLOWER
I20260925 09:24:07.440696 19944 heartbeater.cc:503] Master 127.3.82.62:37625 was elected leader, sending a full tablet report...
I20260925 09:24:07.441525 19960 ts_tablet_manager.cc:1428] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260925 09:24:07.441316 19961 consensus_queue.cc:260] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.442963 19813 heartbeater.cc:503] Master 127.3.82.62:37625 was elected leader, sending a full tablet report...
I20260925 09:24:07.444187 19961 ts_tablet_manager.cc:1428] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Time spent starting tablet: real 0.015s user 0.009s sys 0.008s
I20260925 09:24:07.445742 19962 ts_tablet_manager.cc:1428] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Time spent starting tablet: real 0.015s user 0.005s sys 0.008s
W20260925 09:24:07.515199 19814 tablet.cc:2369] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:24:07.529937 19682 tablet.cc:2369] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:24:07.535617 19945 tablet.cc:2369] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:24:07.540273 19967 raft_consensus.cc:482] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:24:07.540691 19967 raft_consensus.cc:504] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.542513 19967 leader_election.cc:288] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3:46313), 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023)
I20260925 09:24:07.552642 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.543018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:39491 (local address 127.3.82.3:46313)
0925 09:24:07.543220 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.543227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.544105 (+ 878us) server_negotiation.cc:407] Connection header received
0925 09:24:07.544301 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.544306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.544382 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.544485 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:07.545500 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.546720 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.550023 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.551324 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.551478 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.551599 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.552217 (+ 618us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.552362 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:24:07.552646 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.543171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:55189 (local address 127.3.82.1:43023)
0925 09:24:07.543358 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.543365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.544069 (+ 704us) server_negotiation.cc:407] Connection header received
0925 09:24:07.544290 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.544295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.544383 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.544484 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:07.545500 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.546677 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.549948 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.551332 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.551478 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.551592 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.552218 (+ 626us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.552393 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:24:07.552872 19969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.542855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:43023 (local address 127.3.82.2:55189)
0925 09:24:07.543904 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:07.543925 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:07.544106 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:07.544563 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:07.544568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:07.544605 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:07.545294 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.545303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.546859 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.546866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.549714 (+ 2848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.549735 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.551654 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.551659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.551923 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.552048 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 09:24:07.552156 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:24:07.552374 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":260,"thread_start_us":624,"threads_started":1}
I20260925 09:24:07.553355 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.543074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:46313 (local address 127.3.82.2:39491)
0925 09:24:07.543983 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:07.544001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:07.544162 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:07.544569 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:07.544572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:07.544616 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:07.545273 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.545293 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.546859 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.546866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.549714 (+ 2848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:07.549735 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.551636 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.551648 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:07.551925 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.552048 (+ 123us) client_negotiation.cc:769] Sending connection context
0925 09:24:07.552156 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:24:07.552374 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":388,"spinlock_wait_cycles":66304,"thread_start_us":510,"threads_started":1}
I20260925 09:24:07.554004 19636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c979ad80c3045d08dcfb27f16479e95" candidate_uuid: "0e16ceb5871241e5a95f693949484528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" is_pre_election: true
I20260925 09:24:07.554558 19636 raft_consensus.cc:2454] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e16ceb5871241e5a95f693949484528 in term 0.
I20260925 09:24:07.554507 19899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c979ad80c3045d08dcfb27f16479e95" candidate_uuid: "0e16ceb5871241e5a95f693949484528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" is_pre_election: true
I20260925 09:24:07.554992 19899 raft_consensus.cc:2454] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e16ceb5871241e5a95f693949484528 in term 0.
I20260925 09:24:07.555270 19701 leader_election.cc:302] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e16ceb5871241e5a95f693949484528, 86a5649eddd74c30a98ffb2b3e275bb3; no voters:
I20260925 09:24:07.555696 19967 raft_consensus.cc:2781] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:24:07.555850 19967 raft_consensus.cc:482] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:24:07.555950 19967 raft_consensus.cc:3037] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:07.558352 19967 raft_consensus.cc:504] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.559017 19967 leader_election.cc:288] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [CANDIDATE]: Term 1 election: Requested vote from peers 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3:46313), 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023)
I20260925 09:24:07.559577 19899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c979ad80c3045d08dcfb27f16479e95" candidate_uuid: "0e16ceb5871241e5a95f693949484528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a0b6054639e4518bf894fa7ae3a61f4"
I20260925 09:24:07.559633 19636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c979ad80c3045d08dcfb27f16479e95" candidate_uuid: "0e16ceb5871241e5a95f693949484528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
I20260925 09:24:07.559824 19899 raft_consensus.cc:3037] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:07.559855 19636 raft_consensus.cc:3037] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:07.562342 19899 raft_consensus.cc:2454] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e16ceb5871241e5a95f693949484528 in term 1.
I20260925 09:24:07.562759 19636 raft_consensus.cc:2454] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e16ceb5871241e5a95f693949484528 in term 1.
I20260925 09:24:07.562835 19702 leader_election.cc:302] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e16ceb5871241e5a95f693949484528, 1a0b6054639e4518bf894fa7ae3a61f4; no voters:
I20260925 09:24:07.563254 19967 raft_consensus.cc:2781] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:07.563670 19967 raft_consensus.cc:686] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 1 LEADER]: Becoming Leader. State: Replica: 0e16ceb5871241e5a95f693949484528, State: Running, Role: LEADER
I20260925 09:24:07.564191 19967 consensus_queue.cc:237] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } }
I20260925 09:24:07.569936 19493 catalog_manager.cc:5495] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e16ceb5871241e5a95f693949484528 (127.3.82.2). New cstate: current_term: 1 leader_uuid: "0e16ceb5871241e5a95f693949484528" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e16ceb5871241e5a95f693949484528" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 } health_report { overall_health: UNKNOWN } } }
I20260925 09:24:07.590186 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.584242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.3.82.2:42383)
0925 09:24:07.584493 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.584499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.584520 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:24:07.584763 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.584767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.584888 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.585067 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:24:07.585462 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.586315 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.587853 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.588274 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.588419 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.588976 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:24:07.589527 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:24:07.589696 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.589758 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.589874 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:24:07.599234 19899 raft_consensus.cc:1261] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Refusing update from remote peer 0e16ceb5871241e5a95f693949484528: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:07.599236 19636 raft_consensus.cc:1261] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Refusing update from remote peer 0e16ceb5871241e5a95f693949484528: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:07.600389 19971 consensus_queue.cc:1035] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86a5649eddd74c30a98ffb2b3e275bb3" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:07.600785 19967 consensus_queue.cc:1035] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a0b6054639e4518bf894fa7ae3a61f4" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:07.630246 19496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:07.630571 19496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:07.634509 19496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Sending DeleteTablet for 3 replicas of tablet 1c979ad80c3045d08dcfb27f16479e95
I20260925 09:24:07.635823 19879 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c979ad80c3045d08dcfb27f16479e95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:07 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:41450
I20260925 09:24:07.635917 19748 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c979ad80c3045d08dcfb27f16479e95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:07 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:51238
I20260925 09:24:07.636221 19616 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c979ad80c3045d08dcfb27f16479e95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:07 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:32910
I20260925 09:24:07.636699 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19533
I20260925 09:24:07.637787 19984 tablet_replica.cc:330] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: stopping tablet replica
I20260925 09:24:07.638301 19984 raft_consensus.cc:2229] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:07.638885 19984 raft_consensus.cc:2258] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:07.644966 19985 tablet_replica.cc:330] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: stopping tablet replica
I20260925 09:24:07.645526 19985 raft_consensus.cc:2229] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:24:07.646361 19985 raft_consensus.cc:2258] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:07.648041 19986 tablet_replica.cc:330] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: stopping tablet replica
I20260925 09:24:07.648231 19984 ts_tablet_manager.cc:1905] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:07.648526 19986 raft_consensus.cc:2229] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:07.649053 19986 raft_consensus.cc:2258] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:07.651620 19986 ts_tablet_manager.cc:1905] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:07.653496 19985 ts_tablet_manager.cc:1905] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:07.655882 19984 ts_tablet_manager.cc:1918] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:07.656059 19984 log.cc:1196] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/1c979ad80c3045d08dcfb27f16479e95
I20260925 09:24:07.656615 19984 ts_tablet_manager.cc:1939] T 1c979ad80c3045d08dcfb27f16479e95 P 1a0b6054639e4518bf894fa7ae3a61f4: Deleting consensus metadata
I20260925 09:24:07.658694 19481 catalog_manager.cc:4841] TS 1a0b6054639e4518bf894fa7ae3a61f4 (127.3.82.3:46313): tablet 1c979ad80c3045d08dcfb27f16479e95 (table test-table [id=b0550c53e9e348f4a9928e8abb8ec002]) successfully deleted
I20260925 09:24:07.660800 19986 ts_tablet_manager.cc:1918] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:07.660979 19986 log.cc:1196] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/1c979ad80c3045d08dcfb27f16479e95
I20260925 09:24:07.661540 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.638258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.3.82.1:43023)
0925 09:24:07.638494 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.638502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.638525 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:24:07.638585 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.638590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.638692 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.638795 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:07.640986 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.642299 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.644551 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.645233 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.645368 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.651422 (+ 6054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:07.651438 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:07.651463 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:07.651533 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:07.655959 (+ 4426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.657450 (+ 1491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.657461 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.657465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.657701 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.658230 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.658236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.658238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.659543 (+ 1305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:07.659914 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.660620 (+ 706us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.660991 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:24:07.661583 19986 ts_tablet_manager.cc:1939] T 1c979ad80c3045d08dcfb27f16479e95 P 86a5649eddd74c30a98ffb2b3e275bb3: Deleting consensus metadata
I20260925 09:24:07.662569 19676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:07.673946 19985 ts_tablet_manager.cc:1918] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:07.674247 19985 log.cc:1196] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/1c979ad80c3045d08dcfb27f16479e95
I20260925 09:24:07.675060 19985 ts_tablet_manager.cc:1939] T 1c979ad80c3045d08dcfb27f16479e95 P 0e16ceb5871241e5a95f693949484528: Deleting consensus metadata
I20260925 09:24:07.677412 19483 catalog_manager.cc:4841] TS 0e16ceb5871241e5a95f693949484528 (127.3.82.2:42383): tablet 1c979ad80c3045d08dcfb27f16479e95 (table test-table [id=b0550c53e9e348f4a9928e8abb8ec002]) successfully deleted
I20260925 09:24:07.766285 19480 catalog_manager.cc:4841] TS 86a5649eddd74c30a98ffb2b3e275bb3 (127.3.82.1:43023): tablet 1c979ad80c3045d08dcfb27f16479e95 (table test-table [id=b0550c53e9e348f4a9928e8abb8ec002]) successfully deleted
I20260925 09:24:07.766634 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19533
I20260925 09:24:07.791345 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19685
I20260925 09:24:07.810529 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.792223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.3.82.2:42383)
0925 09:24:07.792419 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.792426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.792599 (+ 173us) server_negotiation.cc:407] Connection header received
0925 09:24:07.792800 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.792806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.792900 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.793004 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:07.795003 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.796327 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.798517 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.799219 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.799361 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.803038 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:07.803051 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:07.803074 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:07.803132 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:07.806786 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.808043 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.808050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.808053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.808220 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.808671 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.808675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.808677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.809115 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:07.809449 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.809934 (+ 485us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.810120 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:24:07.811244 19808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:07.930234 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19685
I20260925 09:24:07.949703 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19816
I20260925 09:24:07.964963 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:07.950618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.3.82.3:46313)
0925 09:24:07.950950 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:07.950956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:07.950976 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:24:07.951058 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:07.951062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:07.951128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:07.951213 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:07.952635 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.953419 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.954744 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:07.955152 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:07.955255 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:07.958526 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:07.958542 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:07.958567 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:07.958614 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:07.961489 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.962470 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.962479 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.962482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.962660 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:07.963034 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:07.963037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:07.963040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:07.963440 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:07.963684 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:07.964276 (+ 592us) server_negotiation.cc:299] Negotiation successful
0925 09:24:07.964582 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260925 09:24:07.965718 19939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:08.062032 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19816
I20260925 09:24:08.079411 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 19463
I20260925 09:24:08.080776 19526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:08.199270 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 19463
I20260925 09:24:28.222990 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37625
--webserver_interface=127.3.82.62
--webserver_port=40223
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:28.581238 19991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:28.581642 19991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:28.581840 19991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:28.589470 19991 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 09:24:28.589641 19991 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 09:24:28.589713 19991 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 09:24:28.589792 19991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:24:28.589918 19991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:28.590046 19991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:28.590117 19991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:24:28.590193 19991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:24:28.600519 19991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=40223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:28.601979 19991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:28.603400 19991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:28.616984 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:28.616942 19997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:28.617013 19996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:28.617859 19991 server_base.cc:1034] running on GCE node
I20260925 09:24:28.618657 19991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:28.620204 19991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:28.621474 19991 hybrid_clock.cc:648] HybridClock initialized: now 1790328268621379 us; error 61 us; skew 500 ppm
I20260925 09:24:28.621961 19991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:28 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328268, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:28.625419 19991 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:24:28.627038 19991 webserver.cc:460] Webserver started at http://127.3.82.62:40223/ using document root <none> and password file <none>
I20260925 09:24:28.627803 19991 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:28.627902 19991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:28.633803 19991 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 09:24:28.637424 20006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:28.639263 19991 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260925 09:24:28.639482 19991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "bc335470a63f48318196a52ffed6341d"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "e4686f6c4fb9334c70f49ed5986ccd67"
server_key_iv: "33cee414889439be871f2a68ab0ee2a1"
server_key_version: "encryptionkey@0"
I20260925 09:24:28.640542 19991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:28.658453 19991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:28.663282 19991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:28.663623 19991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:28.683631 19991 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37625
I20260925 09:24:28.683672 20057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37625 every 8 connection(s)
I20260925 09:24:28.685240 19991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:24:28.690330 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 19991
I20260925 09:24:28.691395 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:43023
--local_ip_for_outbound_sockets=127.3.82.1
--tserver_master_addrs=127.3.82.62:37625
--webserver_port=37329
--webserver_interface=127.3.82.1
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:24:28.693607 20058 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:24:28.699131 20058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap starting.
I20260925 09:24:28.720814 20058 log.cc:824] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:28.738039 20058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:24:28.738638 20058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Bootstrap complete.
I20260925 09:24:28.749281 20058 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:28.750419 20058 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc335470a63f48318196a52ffed6341d, State: Initialized, Role: FOLLOWER
I20260925 09:24:28.751233 20058 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:28.751504 20058 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:24:28.751638 20058 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:24:28.751864 20058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:24:28.756172 20058 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:28.756769 20058 leader_election.cc:302] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc335470a63f48318196a52ffed6341d; no voters:
I20260925 09:24:28.757671 20058 leader_election.cc:288] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 09:24:28.757864 20062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 3 FOLLOWER]: Leader election won for term 3
I20260925 09:24:28.759413 20062 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [term 3 LEADER]: Becoming Leader. State: Replica: bc335470a63f48318196a52ffed6341d, State: Running, Role: LEADER
I20260925 09:24:28.760120 20062 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } }
I20260925 09:24:28.760473 20058 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:24:28.763320 20063 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bc335470a63f48318196a52ffed6341d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } } }
I20260925 09:24:28.763736 20063 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:28.763314 20064 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc335470a63f48318196a52ffed6341d. Latest consensus state: current_term: 3 leader_uuid: "bc335470a63f48318196a52ffed6341d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc335470a63f48318196a52ffed6341d" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37625 } } }
I20260925 09:24:28.764272 20064 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:28.768675 20068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:24:28.781347 20068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:24:28.783879 20068 catalog_manager.cc:1259] Loaded cluster ID: 10960b0e7b5349dbb028f475efaa6180
I20260925 09:24:28.784121 20068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:24:28.790318 20068 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:24:28.793205 20068 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Loaded TSK: 0
I20260925 09:24:28.793524 20068 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc335470a63f48318196a52ffed6341d: Loaded TSK: 1
F20260925 09:24:28.795269 20068 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3770feb9ad google::LogMessage::Fail() at ??:0
@ 0x7f3770fee88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3770feb4de google::LogMessage::Flush() at ??:0
@ 0x7f3770feca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3774d4f7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3774d3e73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3786a1e1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3786a1d783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3786aa92a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3786aa90c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3786af35e9 std::function<>::operator()() at ??:0
@ 0x7f3786a21cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3786a1fc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3786aa7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3786aa7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3773c37345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3773c3a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3773c3a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3773c15eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3776eef6db start_thread at ??:0
@ 0x7f376fe8561f clone at ??:0
W20260925 09:24:29.116009 20060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:29.116403 20060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:29.116582 20060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:29.124302 20060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:29.124472 20060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:29.124660 20060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:24:29.134897 20060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:43023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=37329
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:29.136348 20060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:29.137703 20060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:29.155691 20108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:29.155671 20106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:29.155966 20105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:29.160089 20060 server_base.cc:1034] running on GCE node
I20260925 09:24:29.160868 20060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:29.162029 20060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:29.163344 20060 hybrid_clock.cc:648] HybridClock initialized: now 1790328269163220 us; error 72 us; skew 500 ppm
I20260925 09:24:29.163717 20060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:29 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328269, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:29.167537 20060 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:24:29.169623 20060 webserver.cc:460] Webserver started at http://127.3.82.1:37329/ using document root <none> and password file <none>
I20260925 09:24:29.170555 20060 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:29.170714 20060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:29.176703 20060 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:24:29.180069 20115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:29.181764 20060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 09:24:29.182142 20060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "86a5649eddd74c30a98ffb2b3e275bb3"
format_stamp: "Formatted at 2026-09-25 09:23:42 on dist-test-slave-4wm5"
server_key: "aaac84088bd49f5d9bfcf9299bf2dadb"
server_key_iv: "08fc64156c4b716a03cab335e595d19d"
server_key_version: "encryptionkey@0"
I20260925 09:24:29.183224 20060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:29.204367 20060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:29.209295 20060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:29.209607 20060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:29.210700 20060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:29.212144 20060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:29.212270 20060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:29.212422 20060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:29.212512 20060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:29.253938 20060 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:43023
I20260925 09:24:29.254036 20227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:43023 every 8 connection(s)
I20260925 09:24:29.255596 20060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:24:29.260990 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:29.258981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:34909)
0925 09:24:29.259859 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:29.260081 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":384,"threads_started":1}
W20260925 09:24:29.261659 20228 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:37625 (0 consecutive failures): Network error: Failed to ping master at 127.3.82.62:37625: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
I20260925 09:24:29.262733 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:29.262235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:33027)
0925 09:24:29.262464 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:29.262554 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 09:24:29.262714 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20060
I20260925 09:24:29.263422 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:42383
--local_ip_for_outbound_sockets=127.3.82.2
--tserver_master_addrs=127.3.82.62:37625
--webserver_port=38907
--webserver_interface=127.3.82.2
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:24:29.263737 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:29.263296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:36827)
0925 09:24:29.263472 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:29.263573 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260925 09:24:29.264075 20228 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:24:29.614717 20231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:29.615024 20231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:29.615120 20231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:29.622521 20231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:29.622651 20231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:29.622768 20231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:24:29.632795 20231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:42383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=38907
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:29.634162 20231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:29.635507 20231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:29.650331 20239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:29.650513 20236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:29.650393 20237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:29.651384 20231 server_base.cc:1034] running on GCE node
I20260925 09:24:29.652060 20231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:29.653146 20231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:29.654390 20231 hybrid_clock.cc:648] HybridClock initialized: now 1790328269654324 us; error 64 us; skew 500 ppm
I20260925 09:24:29.654739 20231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:29 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328269, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:29.663028 20231 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:24:29.664832 20231 webserver.cc:460] Webserver started at http://127.3.82.2:38907/ using document root <none> and password file <none>
I20260925 09:24:29.665556 20231 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:29.665663 20231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:29.671660 20231 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 09:24:29.674889 20246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:29.676582 20231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 09:24:29.676795 20231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "0e16ceb5871241e5a95f693949484528"
format_stamp: "Formatted at 2026-09-25 09:23:43 on dist-test-slave-4wm5"
server_key: "02bdf4387ce49541578cddb73fe66451"
server_key_iv: "0bbbe861d6df931c48942f9c013f7ec6"
server_key_version: "encryptionkey@0"
I20260925 09:24:29.677989 20231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:29.700373 20231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:29.705130 20231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:29.705423 20231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:29.706610 20231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:29.708253 20231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:29.708340 20231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:29.708472 20231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:29.708593 20231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:29.745215 20231 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:42383
I20260925 09:24:29.745277 20358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:42383 every 8 connection(s)
I20260925 09:24:29.746865 20231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:24:29.752079 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20231
I20260925 09:24:29.752262 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:29.750275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:55749)
0925 09:24:29.751062 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:29.751298 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":289,"threads_started":1}
I20260925 09:24:29.752851 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:46313
--local_ip_for_outbound_sockets=127.3.82.3
--tserver_master_addrs=127.3.82.62:37625
--webserver_port=46765
--webserver_interface=127.3.82.3
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:29.753033 20359 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:37625 (0 consecutive failures): Network error: Failed to ping master at 127.3.82.62:37625: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
I20260925 09:24:29.754061 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:29.753665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:39547)
0925 09:24:29.753820 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:29.753948 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 09:24:29.755095 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:29.754755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:57601)
0925 09:24:29.754910 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:29.754985 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 09:24:29.755379 20359 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 09:24:30.099270 20362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:30.099581 20362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:30.099776 20362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:30.107275 20362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:30.107410 20362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:30.107625 20362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:24:30.117882 20362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:46313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=46765
--tserver_master_addrs=127.3.82.62:37625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:30.119300 20362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:30.120711 20362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:30.134618 20367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:30.134584 20370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:30.134654 20368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:30.135532 20362 server_base.cc:1034] running on GCE node
I20260925 09:24:30.136119 20362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:30.137059 20362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:30.138335 20362 hybrid_clock.cc:648] HybridClock initialized: now 1790328270138282 us; error 57 us; skew 500 ppm
I20260925 09:24:30.138732 20362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:30 dist-test-slave-4wm5 krb5kdc[18912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328270, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:30.142247 20362 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:24:30.143930 20362 webserver.cc:460] Webserver started at http://127.3.82.3:46765/ using document root <none> and password file <none>
I20260925 09:24:30.144762 20362 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:30.144932 20362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:30.151597 20362 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:24:30.154904 20377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:30.156826 20362 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:24:30.157092 20362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "1a0b6054639e4518bf894fa7ae3a61f4"
format_stamp: "Formatted at 2026-09-25 09:23:44 on dist-test-slave-4wm5"
server_key: "2d85f87becf97986e10d42140917aaec"
server_key_iv: "b3d70366660708a558b91ace2b3b64de"
server_key_version: "encryptionkey@0"
I20260925 09:24:30.158448 20362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:30.173183 20362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:30.178629 20362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:30.178982 20362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:30.180239 20362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:30.181969 20362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:30.182166 20362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:30.182350 20362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:30.182464 20362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:30.221873 20362 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:46313
I20260925 09:24:30.222070 20489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:46313 every 8 connection(s)
I20260925 09:24:30.223425 20362 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:24:30.228518 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:30.226579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:53407)
0925 09:24:30.227307 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:30.227545 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":282,"threads_started":1}
W20260925 09:24:30.229334 20490 heartbeater.cc:644] Failed to heartbeat to 127.3.82.62:37625 (0 consecutive failures): Network error: Failed to ping master at 127.3.82.62:37625: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
I20260925 09:24:30.230388 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:30.230056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:43255)
0925 09:24:30.230208 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:30.230279 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 09:24:30.231416 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:30.231111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:53809)
0925 09:24:30.231239 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:30.231311 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:24:30.231765 20490 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 09:24:30.232048 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20362
I20260925 09:24:30.265638 20494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:30.264757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:60629)
0925 09:24:30.265268 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:30.265362 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":259,"threads_started":1}
I20260925 09:24:30.756985 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:30.756146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:33281)
0925 09:24:30.756643 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:30.756736 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260925 09:24:31.233579 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:31.232496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:54323)
0925 09:24:31.233223 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:31.233331 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":409,"threads_started":1}
I20260925 09:24:31.268102 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:31.267100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:48397)
0925 09:24:31.267817 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:31.267913 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":441,"threads_started":1}
I20260925 09:24:31.759227 20498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:31.758346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:50393)
0925 09:24:31.758942 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:31.759044 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":261,"threads_started":1}
I20260925 09:24:32.235395 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:32.234587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:50043)
0925 09:24:32.235072 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:32.235194 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":248,"threads_started":1}
I20260925 09:24:32.270155 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:32.269265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:44857)
0925 09:24:32.269744 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:32.269897 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":272,"threads_started":1}
I20260925 09:24:32.761328 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:32.760477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:35563)
0925 09:24:32.761017 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:32.761129 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":325,"threads_started":1}
I20260925 09:24:33.237277 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:33.236431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:44215)
0925 09:24:33.237004 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:33.237098 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":281,"threads_started":1}
I20260925 09:24:33.272056 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:33.271269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:46129)
0925 09:24:33.271791 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:33.271878 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":284,"threads_started":1}
I20260925 09:24:33.763589 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:33.762630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:50661)
0925 09:24:33.763171 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:33.763261 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":279,"threads_started":1}
I20260925 09:24:34.239430 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:34.238603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:60065)
0925 09:24:34.239140 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:34.239231 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":287,"threads_started":1}
I20260925 09:24:34.274248 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:34.273327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:32793)
0925 09:24:34.273975 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:34.274069 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":259,"threads_started":1}
I20260925 09:24:34.765424 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:34.764627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:39745)
0925 09:24:34.765162 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:34.765249 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":240,"threads_started":1}
I20260925 09:24:35.241447 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:35.240502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:38197)
0925 09:24:35.241158 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:35.241248 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":402,"threads_started":1}
I20260925 09:24:35.276225 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:35.275370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:43937)
0925 09:24:35.275947 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:35.276049 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":270,"threads_started":1}
I20260925 09:24:35.767529 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:35.766559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:42531)
0925 09:24:35.767155 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:35.767297 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":317,"threads_started":1}
I20260925 09:24:36.243739 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:36.242789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:33869)
0925 09:24:36.243414 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:36.243528 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":279,"threads_started":1}
I20260925 09:24:36.278436 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:36.277445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:34499)
0925 09:24:36.278084 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:36.278174 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":249,"threads_started":1}
I20260925 09:24:36.769528 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:36.768635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:33503)
0925 09:24:36.769176 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:36.769271 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":237,"threads_started":1}
I20260925 09:24:37.246001 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:37.244989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:34615)
0925 09:24:37.245535 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:37.245629 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":257,"threads_started":1}
I20260925 09:24:37.280476 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:37.279621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:43535)
0925 09:24:37.280169 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:37.280266 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":320,"threads_started":1}
I20260925 09:24:37.771803 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:37.770845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:41857)
0925 09:24:37.771488 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:37.771605 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":280,"threads_started":1}
I20260925 09:24:38.248021 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:38.247196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:55669)
0925 09:24:38.247708 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:38.247797 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":275,"threads_started":1}
I20260925 09:24:38.282752 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:38.281592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:58593)
0925 09:24:38.282467 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:38.282558 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":586,"threads_started":1}
I20260925 09:24:38.774212 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:38.773108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:43421)
0925 09:24:38.773803 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:38.773957 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":370,"threads_started":1}
I20260925 09:24:39.250049 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:39.249162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:47691)
0925 09:24:39.249691 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:39.249777 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":270,"threads_started":1}
I20260925 09:24:39.284637 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:39.283821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:44261)
0925 09:24:39.284320 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:39.284412 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":268,"threads_started":1}
I20260925 09:24:39.776078 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:39.775290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:51573)
0925 09:24:39.775809 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:39.775895 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":270,"threads_started":1}
I20260925 09:24:40.251991 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:40.251125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:46691)
0925 09:24:40.251649 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:40.251742 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":290,"threads_started":1}
I20260925 09:24:40.286660 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:40.285766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:52537)
0925 09:24:40.286370 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:40.286473 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":332,"threads_started":1}
I20260925 09:24:40.778168 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:40.777266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:49511)
0925 09:24:40.777791 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:40.777981 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":274,"threads_started":1}
I20260925 09:24:41.253958 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:41.253112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:43905)
0925 09:24:41.253656 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:41.253741 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":250,"threads_started":1}
I20260925 09:24:41.288640 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:41.287905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:58541)
0925 09:24:41.288398 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:41.288481 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":272,"threads_started":1}
I20260925 09:24:41.780066 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:41.779207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:35545)
0925 09:24:41.779750 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:41.779840 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":324,"threads_started":1}
I20260925 09:24:42.255970 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:42.255168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:48227)
0925 09:24:42.255693 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:42.255793 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":271,"threads_started":1}
I20260925 09:24:42.290606 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:42.289676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:56877)
0925 09:24:42.290280 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:42.290380 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":325,"threads_started":1}
I20260925 09:24:42.782238 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:42.781216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:46695)
0925 09:24:42.781820 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:42.782022 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":288,"threads_started":1}
I20260925 09:24:43.258041 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:43.257086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:54449)
0925 09:24:43.257672 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:43.257763 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":320,"threads_started":1}
I20260925 09:24:43.292599 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:43.291775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:44097)
0925 09:24:43.292286 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:43.292365 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":290,"threads_started":1}
I20260925 09:24:43.784181 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:43.783363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:48899)
0925 09:24:43.783882 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:43.783978 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":248,"threads_started":1}
I20260925 09:24:44.260140 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:44.259175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:57783)
0925 09:24:44.259818 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:44.259912 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":361,"threads_started":1}
I20260925 09:24:44.294915 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:44.294014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.1:51791)
0925 09:24:44.294641 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:44.294726 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":260,"threads_started":1}
I20260925 09:24:44.786309 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:44.785477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.2:50813)
0925 09:24:44.786009 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:44.786114 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":231,"threads_started":1}
I20260925 09:24:45.237661 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20060
I20260925 09:24:45.258549 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:45.239493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.3.82.1:43023)
0925 09:24:45.240170 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:45.240187 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:45.240224 (+ 37us) server_negotiation.cc:407] Connection header received
0925 09:24:45.242047 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:45.242063 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:45.242548 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:45.242938 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:45.244585 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:45.245996 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:45.247358 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:45.248131 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:45.248354 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:45.251313 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:45.251323 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:45.251341 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:45.251392 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:45.254713 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:45.255605 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:45.255620 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:45.255629 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:45.255825 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:45.256247 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:45.256252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:45.256257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:45.256860 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:45.257219 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:45.257709 (+ 490us) server_negotiation.cc:299] Negotiation successful
0925 09:24:45.258236 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":485,"threads_started":1}
I20260925 09:24:45.260133 20223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:45.262725 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:45.261547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37625 (local address 127.3.82.3:56389)
0925 09:24:45.262236 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:45.262386 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.62:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":317,"threads_started":1}
I20260925 09:24:45.348513 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20060
I20260925 09:24:45.366607 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20231
I20260925 09:24:45.388131 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:45.367675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.3.82.2:42383)
0925 09:24:45.368263 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:45.368277 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:45.368315 (+ 38us) server_negotiation.cc:407] Connection header received
0925 09:24:45.369993 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:45.370010 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:45.370495 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:45.370961 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:45.372500 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:45.373910 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:45.375323 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:45.376116 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:45.376325 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:45.379392 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:45.379406 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:45.379428 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:45.379493 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:45.383805 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:45.384738 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:45.384746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:45.384754 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:45.385046 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:45.385602 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:45.385609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:45.385613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:45.386333 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:45.386574 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:45.387286 (+ 712us) server_negotiation.cc:299] Negotiation successful
0925 09:24:45.387826 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":260,"threads_started":1}
I20260925 09:24:45.389751 20354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:45.481202 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20231
I20260925 09:24:45.501147 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20362
I20260925 09:24:45.521005 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:45.502406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.3.82.3:46313)
0925 09:24:45.503128 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:45.503141 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:45.503181 (+ 40us) server_negotiation.cc:407] Connection header received
0925 09:24:45.504845 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:45.504857 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:45.505365 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:45.505695 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:45.507291 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:45.508651 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:45.510090 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:45.510836 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:45.511017 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:45.514040 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:45.514052 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:45.514071 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:45.514117 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:45.517322 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:45.518251 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:45.518259 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:45.518265 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:45.518470 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:45.518980 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:45.518984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:45.518987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:45.519611 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:45.519887 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:45.520291 (+ 404us) server_negotiation.cc:299] Negotiation successful
0925 09:24:45.520679 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":407,"threads_started":1}
I20260925 09:24:45.522668 20485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:45.609468 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20362
2026-09-25T09:24:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65198 ms)
[----------] 30 tests from SecurityITest (311671 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:24:45 dist-test-slave-4wm5 krb5kdc[20548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:24:45 dist-test-slave-4wm5 krb5kdc[20548](info): set up 2 sockets
Sep 25 09:24:45 dist-test-slave-4wm5 krb5kdc[20548](info): commencing operation
krb5kdc: starting...
W20260925 09:24:46.758538 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:24:46 dist-test-slave-4wm5 krb5kdc[20548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:24:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:24:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:24:47.039178 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36645
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:36645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:47.393184 20561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:47.393613 20561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:47.393869 20561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:47.402482 20561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:24:47.402704 20561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:47.402807 20561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:47.402920 20561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:24:47.402998 20561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:24:47.414686 20561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:36645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:36645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:47.416196 20561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:47.417631 20561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:47.430812 20569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:47.430779 20567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:47.431054 20566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:47.432010 20561 server_base.cc:1034] running on GCE node
I20260925 09:24:47.432905 20561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:47.434711 20561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:47.436044 20561 hybrid_clock.cc:648] HybridClock initialized: now 1790328287435988 us; error 48 us; skew 500 ppm
I20260925 09:24:47.436479 20561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:47 dist-test-slave-4wm5 krb5kdc[20548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:47.440444 20561 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:24:47.442297 20561 webserver.cc:460] Webserver started at http://127.3.82.62:36795/ using document root <none> and password file <none>
I20260925 09:24:47.443121 20561 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:47.443264 20561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:47.443620 20561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:47.447235 20561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "2b1d4486c57c4fae9d39ce64701218f4"
format_stamp: "Formatted at 2026-09-25 09:24:47 on dist-test-slave-4wm5"
server_key: "b9ac194e435ad181282dea0ee39184cf"
server_key_iv: "3985f75fb8cbf7b523755b6a3616f5b0"
server_key_version: "encryptionkey@0"
I20260925 09:24:47.448196 20561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "2b1d4486c57c4fae9d39ce64701218f4"
format_stamp: "Formatted at 2026-09-25 09:24:47 on dist-test-slave-4wm5"
server_key: "b9ac194e435ad181282dea0ee39184cf"
server_key_iv: "3985f75fb8cbf7b523755b6a3616f5b0"
server_key_version: "encryptionkey@0"
I20260925 09:24:47.454566 20561 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:24:47.459074 20576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:47.460880 20561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:24:47.461098 20561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2b1d4486c57c4fae9d39ce64701218f4"
format_stamp: "Formatted at 2026-09-25 09:24:47 on dist-test-slave-4wm5"
server_key: "b9ac194e435ad181282dea0ee39184cf"
server_key_iv: "3985f75fb8cbf7b523755b6a3616f5b0"
server_key_version: "encryptionkey@0"
I20260925 09:24:47.461328 20561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:47.487915 20561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:47.493665 20561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:47.494058 20561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:47.514706 20561 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:36645
I20260925 09:24:47.514739 20627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:36645 every 8 connection(s)
I20260925 09:24:47.516178 20561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:24:47.520325 20628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:47.526512 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20561
I20260925 09:24:47.526743 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:24:47.527408 3400 external_mini_cluster.cc:1428] Setting key 938633646970fbab0207c024c9bbaee5
I20260925 09:24:47.532140 20628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4: Bootstrap starting.
I20260925 09:24:47.536154 20628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:47.537355 20628 log.cc:824] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:47.540768 20628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4: No bootstrap required, opened a new log
Sep 25 09:24:47 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:47.550011 20628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1d4486c57c4fae9d39ce64701218f4" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36645 } }
I20260925 09:24:47.550477 20628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:47.550611 20628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b1d4486c57c4fae9d39ce64701218f4, State: Initialized, Role: FOLLOWER
I20260925 09:24:47.551236 20628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1d4486c57c4fae9d39ce64701218f4" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36645 } }
I20260925 09:24:47.551568 20628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:24:47.551743 20628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:24:47.551947 20628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:47.554080 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:47.529961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52888 (local address 127.3.82.62:36645)
0925 09:24:47.530810 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:47.530823 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:47.530868 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:24:47.532698 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:47.532723 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:47.533140 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:47.533497 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:47.535504 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:47.537130 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:47.538950 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:47.539697 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:47.539845 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:47.544999 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:47.545012 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:47.545030 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:47.545099 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:47.548727 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:47.549597 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:47.549604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:47.549611 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:47.549795 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:47.550272 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:47.550282 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:47.550287 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:47.551070 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:47.551459 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:47.551975 (+ 516us) server_negotiation.cc:299] Negotiation successful
0925 09:24:47.552507 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":397,"threads_started":1}
I20260925 09:24:47.555181 20628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1d4486c57c4fae9d39ce64701218f4" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36645 } }
I20260925 09:24:47.555739 20628 leader_election.cc:302] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b1d4486c57c4fae9d39ce64701218f4; no voters:
I20260925 09:24:47.556411 20628 leader_election.cc:288] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:24:47.556589 20633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:47.557881 20633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [term 1 LEADER]: Becoming Leader. State: Replica: 2b1d4486c57c4fae9d39ce64701218f4, State: Running, Role: LEADER
I20260925 09:24:47.558549 20633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1d4486c57c4fae9d39ce64701218f4" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36645 } }
I20260925 09:24:47.559016 20628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:24:47.561862 20635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b1d4486c57c4fae9d39ce64701218f4. Latest consensus state: current_term: 1 leader_uuid: "2b1d4486c57c4fae9d39ce64701218f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1d4486c57c4fae9d39ce64701218f4" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36645 } } }
I20260925 09:24:47.561843 20634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b1d4486c57c4fae9d39ce64701218f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1d4486c57c4fae9d39ce64701218f4" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 36645 } } }
I20260925 09:24:47.562319 20635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:47.562332 20634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:47.566251 20640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:24:47.571686 20640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:24:47.580396 20650 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:24:47.584133 20640 catalog_manager.cc:1347] Generated new cluster ID: 842112f4945344daad68f649067b43d6
I20260925 09:24:47.584306 20640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:24:47.596405 20640 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:24:47.598114 20640 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:24:47.607496 20640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4: Generated new TSK 0
I20260925 09:24:47.608724 20640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:24:47.686529 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36645
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:48.047354 20654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:48.047710 20654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:48.047855 20654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:48.055630 20654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:48.055753 20654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:48.055949 20654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:24:48.067528 20654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:48.069005 20654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:48.070501 20654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:48.084637 20659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:48.084789 20660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:48.084810 20662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:48.086015 20654 server_base.cc:1034] running on GCE node
I20260925 09:24:48.086678 20654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:48.087723 20654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:48.088958 20654 hybrid_clock.cc:648] HybridClock initialized: now 1790328288088899 us; error 49 us; skew 500 ppm
I20260925 09:24:48.089296 20654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:48 dist-test-slave-4wm5 krb5kdc[20548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328288, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:48.092854 20654 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:24:48.094684 20654 webserver.cc:460] Webserver started at http://127.3.82.1:45581/ using document root <none> and password file <none>
I20260925 09:24:48.095464 20654 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:48.095606 20654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:48.095943 20654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:48.099179 20654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "a7ef42ac1ff648009741dd9634294b96"
format_stamp: "Formatted at 2026-09-25 09:24:48 on dist-test-slave-4wm5"
server_key: "1b4e85c9c56331aeaa6c7cd2fd047a31"
server_key_iv: "2bea776eb6635423a9a6e976c628cd20"
server_key_version: "encryptionkey@0"
I20260925 09:24:48.100049 20654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "a7ef42ac1ff648009741dd9634294b96"
format_stamp: "Formatted at 2026-09-25 09:24:48 on dist-test-slave-4wm5"
server_key: "1b4e85c9c56331aeaa6c7cd2fd047a31"
server_key_iv: "2bea776eb6635423a9a6e976c628cd20"
server_key_version: "encryptionkey@0"
I20260925 09:24:48.106086 20654 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:24:48.110184 20669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:48.111964 20654 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:24:48.112176 20654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "a7ef42ac1ff648009741dd9634294b96"
format_stamp: "Formatted at 2026-09-25 09:24:48 on dist-test-slave-4wm5"
server_key: "1b4e85c9c56331aeaa6c7cd2fd047a31"
server_key_iv: "2bea776eb6635423a9a6e976c628cd20"
server_key_version: "encryptionkey@0"
I20260925 09:24:48.112406 20654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:48.125125 20654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:48.130088 20654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:48.130477 20654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:48.131486 20654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:48.132879 20654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:48.132983 20654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:48.133142 20654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:48.133215 20654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:48.171139 20654 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:41237
I20260925 09:24:48.171196 20781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:41237 every 8 connection(s)
I20260925 09:24:48.172611 20654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:24:48.177691 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20654
I20260925 09:24:48.177963 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:24:48.178817 3400 external_mini_cluster.cc:1428] Setting key 3164afe3ef491b84804656f8d72e501b
Sep 25 09:24:48 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328288, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:48.196671 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:48.175340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:46671 (local address 127.3.82.62:36645)
0925 09:24:48.175833 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:48.175841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:48.176924 (+ 1083us) server_negotiation.cc:407] Connection header received
0925 09:24:48.179864 (+ 2940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:48.179869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:48.179972 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:48.180177 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:48.182259 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.183062 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.184854 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.185312 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.185494 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:48.189682 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:48.189689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:48.189692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:48.189729 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:48.192797 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:48.193780 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:48.193785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:48.193788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:48.194010 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:48.194599 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:48.194604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:48.194606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:48.194910 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:48.195092 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:48.195938 (+ 846us) server_negotiation.cc:299] Negotiation successful
0925 09:24:48.196209 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
I20260925 09:24:48.197778 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:48.175925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36645 (local address 127.3.82.1:46671)
0925 09:24:48.176733 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:48.176774 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:48.179492 (+ 2718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:48.180476 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:48.180486 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:48.181270 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:48.182036 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:48.182061 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.183273 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.183278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:48.184645 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:48.184662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.185606 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.185616 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:48.185977 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:48.186765 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:48.186789 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:48.189381 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:48.193022 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:48.193042 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:48.193059 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:48.193603 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:48.194147 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:48.194153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:48.194157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:48.194436 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:48.195063 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:48.195073 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:48.195493 (+ 420us) client_negotiation.cc:769] Sending connection context
0925 09:24:48.195802 (+ 309us) client_negotiation.cc:240] Negotiation successful
0925 09:24:48.196467 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":313,"threads_started":1}
I20260925 09:24:48.199795 20782 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36645
I20260925 09:24:48.200169 20782 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:48.201009 20782 heartbeater.cc:511] Master 127.3.82.62:36645 requested a full tablet report, sending...
I20260925 09:24:48.203823 20593 ts_manager.cc:194] Registered new tserver with Master: a7ef42ac1ff648009741dd9634294b96 (127.3.82.1:41237)
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:24:48.205521 20593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:46671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:24:48.242388 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36645
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:48.608677 20788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:48.609004 20788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:48.609110 20788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:48.617314 20788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:48.617458 20788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:48.617655 20788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:24:48.628229 20788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:48.629709 20788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:48.631167 20788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:48.646492 20793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:48.646435 20794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:48.646402 20796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:48.647727 20788 server_base.cc:1034] running on GCE node
I20260925 09:24:48.648615 20788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:48.649916 20788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:48.651165 20788 hybrid_clock.cc:648] HybridClock initialized: now 1790328288651116 us; error 48 us; skew 500 ppm
I20260925 09:24:48.651611 20788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:48 dist-test-slave-4wm5 krb5kdc[20548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328288, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:48.655512 20788 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:24:48.657315 20788 webserver.cc:460] Webserver started at http://127.3.82.2:44041/ using document root <none> and password file <none>
I20260925 09:24:48.658250 20788 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:48.658421 20788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:48.658764 20788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:48.662196 20788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "0ff53c06a8164a819bdd8c375ffe1d10"
format_stamp: "Formatted at 2026-09-25 09:24:48 on dist-test-slave-4wm5"
server_key: "95b41b9e456a85f2f035fe6b03534db5"
server_key_iv: "6d7a65d67eeb9785c7b459e4206a81d8"
server_key_version: "encryptionkey@0"
I20260925 09:24:48.663185 20788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "0ff53c06a8164a819bdd8c375ffe1d10"
format_stamp: "Formatted at 2026-09-25 09:24:48 on dist-test-slave-4wm5"
server_key: "95b41b9e456a85f2f035fe6b03534db5"
server_key_iv: "6d7a65d67eeb9785c7b459e4206a81d8"
server_key_version: "encryptionkey@0"
I20260925 09:24:48.669315 20788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:24:48.673707 20803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:48.675549 20788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:24:48.675783 20788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "0ff53c06a8164a819bdd8c375ffe1d10"
format_stamp: "Formatted at 2026-09-25 09:24:48 on dist-test-slave-4wm5"
server_key: "95b41b9e456a85f2f035fe6b03534db5"
server_key_iv: "6d7a65d67eeb9785c7b459e4206a81d8"
server_key_version: "encryptionkey@0"
I20260925 09:24:48.676107 20788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:48.696807 20788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:48.701942 20788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:48.702272 20788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:48.703312 20788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:48.704789 20788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:48.705017 20788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:48.705181 20788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:48.705257 20788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:48.742614 20788 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:42449
I20260925 09:24:48.742710 20915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:42449 every 8 connection(s)
I20260925 09:24:48.744050 20788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:24:48.751590 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20788
I20260925 09:24:48.751823 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:24:48.752331 3400 external_mini_cluster.cc:1428] Setting key bf9e31b46f40afd8da1fd4412979679f
Sep 25 09:24:48 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328288, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:48.768270 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:48.747032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:56447 (local address 127.3.82.62:36645)
0925 09:24:48.747567 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:48.747572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:48.748444 (+ 872us) server_negotiation.cc:407] Connection header received
0925 09:24:48.750575 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:48.750580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:48.750679 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:48.750804 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:48.752635 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.753575 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.755563 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.756019 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.756175 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:48.760743 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:48.760751 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:48.760754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:48.760793 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:48.764010 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:48.765173 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:48.765177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:48.765180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:48.765353 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:48.766022 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:48.766027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:48.766029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:48.766445 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:48.766623 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:48.767722 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0925 09:24:48.767944 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":266,"threads_started":1}
I20260925 09:24:48.770011 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:48.747404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36645 (local address 127.3.82.2:56447)
0925 09:24:48.748263 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:48.748306 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:48.750284 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:48.751073 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:48.751081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:48.751635 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:48.752426 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:48.752460 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.753728 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.753734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:48.755291 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:48.755304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:48.756262 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:48.756269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:48.756542 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:48.757422 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:48.757455 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:48.760418 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:48.764214 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:48.764228 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:48.764252 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:48.765008 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:48.765532 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:48.765540 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:48.765556 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:48.765874 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:48.766624 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:48.766679 (+ 55us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:48.767244 (+ 565us) client_negotiation.cc:769] Sending connection context
0925 09:24:48.767624 (+ 380us) client_negotiation.cc:240] Negotiation successful
0925 09:24:48.768389 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":398,"threads_started":1}
I20260925 09:24:48.771811 20916 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36645
I20260925 09:24:48.772185 20916 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:48.773042 20916 heartbeater.cc:511] Master 127.3.82.62:36645 requested a full tablet report, sending...
I20260925 09:24:48.774760 20593 ts_manager.cc:194] Registered new tserver with Master: 0ff53c06a8164a819bdd8c375ffe1d10 (127.3.82.2:42449)
I20260925 09:24:48.775652 20593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:56447
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:24:48.818313 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36645
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:49.179536 20922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:49.179945 20922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:49.180058 20922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:49.187860 20922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:49.188000 20922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:49.188118 20922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:24:49.198527 20922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:36645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:49.199862 20922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:49.201227 20922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:24:49.209102 20782 heartbeater.cc:503] Master 127.3.82.62:36645 was elected leader, sending a full tablet report...
W20260925 09:24:49.216881 20927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:49.216749 20930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:49.216852 20928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:49.218098 20922 server_base.cc:1034] running on GCE node
I20260925 09:24:49.218881 20922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:49.220139 20922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:49.221441 20922 hybrid_clock.cc:648] HybridClock initialized: now 1790328289221427 us; error 93 us; skew 500 ppm
I20260925 09:24:49.221967 20922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:49 dist-test-slave-4wm5 krb5kdc[20548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328289, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:49.225926 20922 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:24:49.227660 20922 webserver.cc:460] Webserver started at http://127.3.82.3:37505/ using document root <none> and password file <none>
I20260925 09:24:49.228458 20922 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:49.228564 20922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:49.228931 20922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:49.232169 20922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "0532926234c742cbbba6eed7b0928aeb"
format_stamp: "Formatted at 2026-09-25 09:24:49 on dist-test-slave-4wm5"
server_key: "f1eddae526fb69023fe36d7d222677e7"
server_key_iv: "8b5530d75a8f7928e1ffa37ff2c09929"
server_key_version: "encryptionkey@0"
I20260925 09:24:49.233057 20922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "0532926234c742cbbba6eed7b0928aeb"
format_stamp: "Formatted at 2026-09-25 09:24:49 on dist-test-slave-4wm5"
server_key: "f1eddae526fb69023fe36d7d222677e7"
server_key_iv: "8b5530d75a8f7928e1ffa37ff2c09929"
server_key_version: "encryptionkey@0"
I20260925 09:24:49.239199 20922 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260925 09:24:49.243827 20937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:49.245673 20922 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 09:24:49.245973 20922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "0532926234c742cbbba6eed7b0928aeb"
format_stamp: "Formatted at 2026-09-25 09:24:49 on dist-test-slave-4wm5"
server_key: "f1eddae526fb69023fe36d7d222677e7"
server_key_iv: "8b5530d75a8f7928e1ffa37ff2c09929"
server_key_version: "encryptionkey@0"
I20260925 09:24:49.246227 20922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:49.260311 20922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:49.265672 20922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:49.266033 20922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:49.267221 20922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:49.268766 20922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:49.268875 20922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:49.268988 20922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:49.269040 20922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:49.306197 20922 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:42333
I20260925 09:24:49.306257 21049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:42333 every 8 connection(s)
I20260925 09:24:49.307621 20922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:24:49.316269 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 20922
I20260925 09:24:49.316468 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:24:49.317211 3400 external_mini_cluster.cc:1428] Setting key dbc7f0cf0cd1432815c94757080c5dcd
Sep 25 09:24:49 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328289, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:49.333252 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.310723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:60013 (local address 127.3.82.62:36645)
0925 09:24:49.311246 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.311250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.312413 (+ 1163us) server_negotiation.cc:407] Connection header received
0925 09:24:49.314529 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.314534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.314616 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.314727 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:49.317054 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.318382 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.320668 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.321220 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.321339 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.325395 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:49.325403 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:49.325407 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:49.325452 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:49.329050 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.330146 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.330151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.330154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.330299 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.330907 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.330913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.330915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.331343 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:49.331516 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.332598 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.332906 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":289,"threads_started":1}
I20260925 09:24:49.334487 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.311329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:36645 (local address 127.3.82.3:60013)
0925 09:24:49.312207 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:49.312262 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:49.314154 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:49.315112 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:49.315122 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:49.315739 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:49.316726 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.316755 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.318604 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.318608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.320440 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.320456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.321467 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.321473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.321698 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.322564 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:49.322589 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:49.325054 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:49.329284 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:49.329292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:49.329309 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:49.329817 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:49.330426 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:49.330431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:49.330434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:49.330674 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:49.331516 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:49.331523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:49.332022 (+ 499us) client_negotiation.cc:769] Sending connection context
0925 09:24:49.332372 (+ 350us) client_negotiation.cc:240] Negotiation successful
0925 09:24:49.332916 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":309,"threads_started":1}
I20260925 09:24:49.336301 21050 heartbeater.cc:347] Connected to a master server at 127.3.82.62:36645
I20260925 09:24:49.336712 21050 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:49.337633 21050 heartbeater.cc:511] Master 127.3.82.62:36645 requested a full tablet report, sending...
I20260925 09:24:49.339623 20593 ts_manager.cc:194] Registered new tserver with Master: 0532926234c742cbbba6eed7b0928aeb (127.3.82.3:42333)
I20260925 09:24:49.340515 20593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:60013
I20260925 09:24:49.347756 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:24:49.368674 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.353225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.3.82.62:36645)
0925 09:24:49.353530 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.353537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.353876 (+ 339us) server_negotiation.cc:407] Connection header received
0925 09:24:49.354146 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.354150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.354222 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.354328 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:49.355873 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.356715 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.358294 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.358830 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.358950 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.362061 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:49.362067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:49.362071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:49.362112 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:49.365103 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.366315 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.366320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.366322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.366469 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.366976 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.366979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.366981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.367328 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:49.367562 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.368057 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.368248 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260925 09:24:49.374241 20593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:24:49.382565 20593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:24:49.410593 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.397716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.3.82.1:41237)
0925 09:24:49.398758 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.398775 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.399665 (+ 890us) server_negotiation.cc:407] Connection header received
0925 09:24:49.399781 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.399790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.400119 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.400326 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:49.401489 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.403628 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.407246 (+ 3618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.408795 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.408952 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.409235 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.410012 (+ 777us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.410203 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":2314,"threads_started":1}
I20260925 09:24:49.411455 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.399445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.3.82.3:42333)
0925 09:24:49.401313 (+ 1868us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.401323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.401976 (+ 653us) server_negotiation.cc:407] Connection header received
0925 09:24:49.402087 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.402095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.402373 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.402600 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:49.403247 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.404797 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.407624 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.409073 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.409212 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.409469 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.410899 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.411148 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1644,"thread_start_us":2817,"threads_started":1}
I20260925 09:24:49.411773 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.399876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:42333 (local address 127.0.0.1:54034)
0925 09:24:49.401630 (+ 1754us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:49.401651 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:49.401781 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:49.402580 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:49.402585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:49.402613 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:49.403037 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.403048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.405013 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.405020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.407421 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.407437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.409631 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.409637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.409860 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.409903 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 09:24:49.409992 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 09:24:49.410215 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1625,"mutex_wait_us":1,"thread_start_us":1099,"threads_started":1}
I20260925 09:24:49.412426 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.396936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:41237 (local address 127.0.0.1:45406)
0925 09:24:49.399221 (+ 2285us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:49.399267 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:49.399456 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:49.400670 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:49.400681 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:49.400731 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:49.401230 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.401247 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.403817 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.403824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.407070 (+ 3246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.407081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.409397 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.409405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.409652 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.409720 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 09:24:49.410926 (+ 1206us) client_negotiation.cc:240] Negotiation successful
0925 09:24:49.411087 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2044,"thread_start_us":1881,"threads_started":1}
I20260925 09:24:49.415313 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.400338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42449 (local address 127.0.0.1:44486)
0925 09:24:49.404336 (+ 3998us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:49.404359 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:49.404493 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:49.405911 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:49.405916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:49.405947 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:49.406383 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.406394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.408303 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.408311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.411788 (+ 3477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.411813 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.414508 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.414517 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.414649 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.414674 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:24:49.414799 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 09:24:49.414956 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3840,"thread_start_us":3673,"threads_started":1}
I20260925 09:24:49.415364 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.399954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.3.82.2:42449)
0925 09:24:49.405035 (+ 5081us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.405043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.405073 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:24:49.405163 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.405171 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.405431 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.405636 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:49.406594 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.408145 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.412236 (+ 4091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.413878 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.414049 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.414264 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.414866 (+ 602us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.415031 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4922,"thread_start_us":4756,"threads_started":1}
I20260925 09:24:49.416464 20985 tablet_service.cc:1467] Processing CreateTablet for tablet 579a85e388f74b589e5dff06dd246014 (DEFAULT_TABLE table=test-table [id=622a08af6e5f419e9899d719570d4717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:49.416456 20717 tablet_service.cc:1467] Processing CreateTablet for tablet 579a85e388f74b589e5dff06dd246014 (DEFAULT_TABLE table=test-table [id=622a08af6e5f419e9899d719570d4717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:49.418179 20985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 579a85e388f74b589e5dff06dd246014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:49.418179 20717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 579a85e388f74b589e5dff06dd246014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:49.419004 20851 tablet_service.cc:1467] Processing CreateTablet for tablet 579a85e388f74b589e5dff06dd246014 (DEFAULT_TABLE table=test-table [id=622a08af6e5f419e9899d719570d4717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:49.420238 20851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 579a85e388f74b589e5dff06dd246014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:49.432391 21069 tablet_bootstrap.cc:492] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Bootstrap starting.
I20260925 09:24:49.436703 21069 tablet_bootstrap.cc:654] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:49.437808 21069 log.cc:824] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:49.439141 21071 tablet_bootstrap.cc:492] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Bootstrap starting.
I20260925 09:24:49.440073 21070 tablet_bootstrap.cc:492] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Bootstrap starting.
I20260925 09:24:49.442696 21069 tablet_bootstrap.cc:492] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: No bootstrap required, opened a new log
I20260925 09:24:49.443020 21069 ts_tablet_manager.cc:1397] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260925 09:24:49.444310 21070 tablet_bootstrap.cc:654] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:49.445003 21071 tablet_bootstrap.cc:654] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:49.445817 21070 log.cc:824] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:49.446355 21071 log.cc:824] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:49.449298 21070 tablet_bootstrap.cc:492] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: No bootstrap required, opened a new log
I20260925 09:24:49.449709 21070 ts_tablet_manager.cc:1397] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260925 09:24:49.449779 21071 tablet_bootstrap.cc:492] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: No bootstrap required, opened a new log
I20260925 09:24:49.450150 21071 ts_tablet_manager.cc:1397] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260925 09:24:49.453548 21069 raft_consensus.cc:348] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.454135 21069 raft_consensus.cc:374] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:49.454252 21069 raft_consensus.cc:729] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff53c06a8164a819bdd8c375ffe1d10, State: Initialized, Role: FOLLOWER
I20260925 09:24:49.455019 21069 consensus_queue.cc:260] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.456305 20916 heartbeater.cc:503] Master 127.3.82.62:36645 was elected leader, sending a full tablet report...
I20260925 09:24:49.457150 21069 ts_tablet_manager.cc:1428] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260925 09:24:49.459530 21070 raft_consensus.cc:348] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.460011 21070 raft_consensus.cc:374] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:49.460127 21070 raft_consensus.cc:729] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ef42ac1ff648009741dd9634294b96, State: Initialized, Role: FOLLOWER
I20260925 09:24:49.460812 21070 consensus_queue.cc:260] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.461508 21071 raft_consensus.cc:348] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.461979 21071 raft_consensus.cc:374] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:49.462110 21071 raft_consensus.cc:729] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0532926234c742cbbba6eed7b0928aeb, State: Initialized, Role: FOLLOWER
I20260925 09:24:49.462821 21071 consensus_queue.cc:260] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.464043 21050 heartbeater.cc:503] Master 127.3.82.62:36645 was elected leader, sending a full tablet report...
I20260925 09:24:49.465016 21071 ts_tablet_manager.cc:1428] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260925 09:24:49.467912 21070 ts_tablet_manager.cc:1428] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260925 09:24:49.482169 21076 raft_consensus.cc:482] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:24:49.482502 21076 raft_consensus.cc:504] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.484501 21076 leader_election.cc:288] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ff53c06a8164a819bdd8c375ffe1d10 (127.3.82.2:42449), 0532926234c742cbbba6eed7b0928aeb (127.3.82.3:42333)
I20260925 09:24:49.494606 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.485810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:42469 (local address 127.3.82.3:42333)
0925 09:24:49.486115 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.486123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.486490 (+ 367us) server_negotiation.cc:407] Connection header received
0925 09:24:49.486748 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.486754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.486866 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.486969 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:49.487621 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.488938 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.491879 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.493228 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.493399 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.493511 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.494178 (+ 667us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.494328 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260925 09:24:49.494606 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.485060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:35159 (local address 127.3.82.2:42449)
0925 09:24:49.485338 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.485346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.485936 (+ 590us) server_negotiation.cc:407] Connection header received
0925 09:24:49.486394 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.486401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.486476 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.486645 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:49.487344 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.488610 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.491848 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.493228 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.493399 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.493511 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.494178 (+ 667us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.494356 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260925 09:24:49.494837 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.485344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:42333 (local address 127.3.82.1:42469)
0925 09:24:49.486387 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:49.486401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:49.486544 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:49.487010 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:49.487017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:49.487049 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:49.487432 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.487440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.489090 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.489100 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.491608 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.491624 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.493589 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.493599 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.493924 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.493967 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 09:24:49.494091 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 09:24:49.494291 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":124,"thread_start_us":524,"threads_started":1}
I20260925 09:24:49.495358 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.484758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:42449 (local address 127.3.82.1:35159)
0925 09:24:49.485775 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:49.485793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:49.486116 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:49.486621 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:49.486638 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:49.486671 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:49.487109 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.487123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.488774 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.488778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.491604 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:49.491624 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.493607 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.493613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:49.493996 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.494035 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 09:24:49.494123 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 09:24:49.494291 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":45,"thread_start_us":453,"threads_started":1}
W20260925 09:24:49.496265 20917 tablet.cc:2369] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:24:49.496106 21005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "579a85e388f74b589e5dff06dd246014" candidate_uuid: "a7ef42ac1ff648009741dd9634294b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0532926234c742cbbba6eed7b0928aeb" is_pre_election: true
I20260925 09:24:49.496573 20871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "579a85e388f74b589e5dff06dd246014" candidate_uuid: "a7ef42ac1ff648009741dd9634294b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" is_pre_election: true
I20260925 09:24:49.496724 21005 raft_consensus.cc:2454] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ef42ac1ff648009741dd9634294b96 in term 0.
I20260925 09:24:49.497188 20871 raft_consensus.cc:2454] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ef42ac1ff648009741dd9634294b96 in term 0.
I20260925 09:24:49.497548 20672 leader_election.cc:302] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0532926234c742cbbba6eed7b0928aeb, a7ef42ac1ff648009741dd9634294b96; no voters:
I20260925 09:24:49.498003 21076 raft_consensus.cc:2781] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:24:49.498180 21076 raft_consensus.cc:482] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:24:49.498324 21076 raft_consensus.cc:3037] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:49.500787 21076 raft_consensus.cc:504] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.501482 21076 leader_election.cc:288] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [CANDIDATE]: Term 1 election: Requested vote from peers 0ff53c06a8164a819bdd8c375ffe1d10 (127.3.82.2:42449), 0532926234c742cbbba6eed7b0928aeb (127.3.82.3:42333)
I20260925 09:24:49.502207 21005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "579a85e388f74b589e5dff06dd246014" candidate_uuid: "a7ef42ac1ff648009741dd9634294b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0532926234c742cbbba6eed7b0928aeb"
I20260925 09:24:49.502206 20871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "579a85e388f74b589e5dff06dd246014" candidate_uuid: "a7ef42ac1ff648009741dd9634294b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff53c06a8164a819bdd8c375ffe1d10"
I20260925 09:24:49.502521 21005 raft_consensus.cc:3037] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:49.502561 20871 raft_consensus.cc:3037] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:49.506538 20871 raft_consensus.cc:2454] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ef42ac1ff648009741dd9634294b96 in term 1.
I20260925 09:24:49.506538 21005 raft_consensus.cc:2454] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ef42ac1ff648009741dd9634294b96 in term 1.
I20260925 09:24:49.507238 20672 leader_election.cc:302] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0532926234c742cbbba6eed7b0928aeb, a7ef42ac1ff648009741dd9634294b96; no voters:
I20260925 09:24:49.507648 21076 raft_consensus.cc:2781] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:49.508188 21076 raft_consensus.cc:686] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 1 LEADER]: Becoming Leader. State: Replica: a7ef42ac1ff648009741dd9634294b96, State: Running, Role: LEADER
I20260925 09:24:49.508764 21076 consensus_queue.cc:237] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } }
I20260925 09:24:49.513731 20593 catalog_manager.cc:5495] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7ef42ac1ff648009741dd9634294b96 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "a7ef42ac1ff648009741dd9634294b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ef42ac1ff648009741dd9634294b96" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 41237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20260925 09:24:49.556630 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.550685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.3.82.1:41237)
0925 09:24:49.550990 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.550998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.551053 (+ 55us) server_negotiation.cc:407] Connection header received
0925 09:24:49.551367 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.551372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.551456 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.551623 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:24:49.552101 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.552853 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.554390 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.554803 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.554947 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.555491 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:24:49.556020 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:24:49.556192 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.556251 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.556349 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260925 09:24:49.559544 21051 tablet.cc:2369] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:24:49.566339 20871 raft_consensus.cc:1261] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 1 FOLLOWER]: Refusing update from remote peer a7ef42ac1ff648009741dd9634294b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:49.566326 21005 raft_consensus.cc:1261] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 1 FOLLOWER]: Refusing update from remote peer a7ef42ac1ff648009741dd9634294b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:49.567406 21080 consensus_queue.cc:1035] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0532926234c742cbbba6eed7b0928aeb" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:49.567816 21076 consensus_queue.cc:1035] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ff53c06a8164a819bdd8c375ffe1d10" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:49.590155 21085 mvcc.cc:205] Tried to move back new op lower bound from 7333184674050609152 to 7333184673832263680. Current Snapshot: MvccSnapshot[applied={T|T < 7333184674050609152}]
I20260925 09:24:49.599046 20592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:49.599483 20592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:49.605873 20592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b1d4486c57c4fae9d39ce64701218f4: Sending DeleteTablet for 3 replicas of tablet 579a85e388f74b589e5dff06dd246014
I20260925 09:24:49.609472 20851 tablet_service.cc:1514] Processing DeleteTablet for tablet 579a85e388f74b589e5dff06dd246014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:49 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:44486
I20260925 09:24:49.609869 20850 tablet_service.cc:1514] Processing DeleteTablet for tablet 579a85e388f74b589e5dff06dd246014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:49 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:44486
I20260925 09:24:49.610121 20717 tablet_service.cc:1514] Processing DeleteTablet for tablet 579a85e388f74b589e5dff06dd246014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:49 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:45406
I20260925 09:24:49.610291 20985 tablet_service.cc:1514] Processing DeleteTablet for tablet 579a85e388f74b589e5dff06dd246014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:49 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:54034
I20260925 09:24:49.611698 21094 tablet_replica.cc:330] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: stopping tablet replica
I20260925 09:24:49.612151 21094 raft_consensus.cc:2229] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:49.612689 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20654
I20260925 09:24:49.612706 21094 raft_consensus.cc:2258] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:49.613693 21095 tablet_replica.cc:330] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: stopping tablet replica
I20260925 09:24:49.614144 21095 raft_consensus.cc:2229] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:49.614645 21095 raft_consensus.cc:2258] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:49.615479 21093 tablet_replica.cc:330] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: stopping tablet replica
I20260925 09:24:49.615973 21093 raft_consensus.cc:2229] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:24:49.616811 21094 ts_tablet_manager.cc:1905] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:49.616858 21093 raft_consensus.cc:2258] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:49.619710 21093 ts_tablet_manager.cc:1905] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:49.624475 21094 ts_tablet_manager.cc:1918] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:49.624671 21094 log.cc:1196] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/579a85e388f74b589e5dff06dd246014
I20260925 09:24:49.625263 21094 ts_tablet_manager.cc:1939] T 579a85e388f74b589e5dff06dd246014 P 0532926234c742cbbba6eed7b0928aeb: Deleting consensus metadata
I20260925 09:24:49.626542 21095 ts_tablet_manager.cc:1905] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:49.627346 20579 catalog_manager.cc:4841] TS 0532926234c742cbbba6eed7b0928aeb (127.3.82.3:42333): tablet 579a85e388f74b589e5dff06dd246014 (table test-table [id=622a08af6e5f419e9899d719570d4717]) successfully deleted
Sep 25 09:24:49 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:24:49.631850 21093 ts_tablet_manager.cc:1918] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:49.632079 21093 log.cc:1196] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/579a85e388f74b589e5dff06dd246014
I20260925 09:24:49.632649 21093 ts_tablet_manager.cc:1939] T 579a85e388f74b589e5dff06dd246014 P a7ef42ac1ff648009741dd9634294b96: Deleting consensus metadata
I20260925 09:24:49.634797 20580 catalog_manager.cc:4841] TS a7ef42ac1ff648009741dd9634294b96 (127.3.82.1:41237): tablet 579a85e388f74b589e5dff06dd246014 (table test-table [id=622a08af6e5f419e9899d719570d4717]) successfully deleted
I20260925 09:24:49.635818 21095 ts_tablet_manager.cc:1918] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:49.636029 21095 log.cc:1196] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/579a85e388f74b589e5dff06dd246014
I20260925 09:24:49.636631 21095 ts_tablet_manager.cc:1939] T 579a85e388f74b589e5dff06dd246014 P 0ff53c06a8164a819bdd8c375ffe1d10: Deleting consensus metadata
I20260925 09:24:49.638521 20580 catalog_manager.cc:4841] TS 0ff53c06a8164a819bdd8c375ffe1d10 (127.3.82.2:42449): tablet 579a85e388f74b589e5dff06dd246014 (table test-table [id=622a08af6e5f419e9899d719570d4717]) successfully deleted
W20260925 09:24:49.639151 20580 catalog_manager.cc:4808] TS 0ff53c06a8164a819bdd8c375ffe1d10 (127.3.82.2:42449): delete failed for tablet 579a85e388f74b589e5dff06dd246014 because the tablet was not found. No further retry: Not found: Tablet not found: 579a85e388f74b589e5dff06dd246014
I20260925 09:24:49.640182 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.615758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.3.82.1:41237)
0925 09:24:49.616236 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.616244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.617893 (+ 1649us) server_negotiation.cc:407] Connection header received
0925 09:24:49.618221 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.618226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.618321 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.618398 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:49.620203 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.621468 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.623577 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.624323 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.624439 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.630477 (+ 6038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:49.630494 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:49.630515 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:49.630570 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:49.635579 (+ 5009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.636865 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.636882 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.636886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.637075 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.637552 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.637558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.637562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.638101 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:49.638442 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.639593 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.639859 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351}
I20260925 09:24:49.640822 20777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:49.742753 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20654
I20260925 09:24:49.768461 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20788
Sep 25 09:24:49 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:24:49.800058 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.769477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.3.82.2:42449)
0925 09:24:49.769672 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.769680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.769711 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:24:49.770066 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.770072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.770152 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.770272 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:49.771687 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.772952 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.774723 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.775182 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.775304 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.793088 (+ 17784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:49.793107 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:49.793123 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:49.793173 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:49.796463 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.797563 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.797572 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.797574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.797734 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.798298 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.798302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.798305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.798788 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:49.799152 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.799467 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.799670 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:24:49.800812 20911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:49.897078 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20788
I20260925 09:24:49.914798 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20922
Sep 25 09:24:49 dist-test-slave-4wm5 krb5kdc[20548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:24:49.932314 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:49.915625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.3.82.3:42333)
0925 09:24:49.915802 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:49.915806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:49.915826 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:24:49.916106 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:49.916110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:49.916199 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:49.916358 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:49.917713 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.918563 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.920031 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:49.920459 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:49.920604 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:49.925563 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:49.925581 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:49.925599 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:49.925646 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:49.928795 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.929684 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.929692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.929694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.930113 (+ 419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:49.930616 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:49.930621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:49.930623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:49.931085 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:49.931314 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:49.931769 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 09:24:49.932040 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 09:24:49.933085 21045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:50.028597 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20922
I20260925 09:24:50.046159 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 20561
I20260925 09:24:50.047502 20623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:50.162058 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 20561
2026-09-25T09:24:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4556 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T09:24:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:24:50Z Disabled control of system clock
I20260925 09:24:50.246927 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37531
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:24:50.600329 21104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:50.600660 21104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:50.600745 21104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:50.608325 21104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:24:50.608454 21104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:50.608492 21104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:50.608542 21104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:24:50.608587 21104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:24:50.618448 21104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:50.619794 21104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:50.621574 21104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:50.634476 21109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:50.634539 21110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:50.634531 21112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:50.636013 21104 server_base.cc:1034] running on GCE node
I20260925 09:24:50.637135 21104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:50.639005 21104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:50.640257 21104 hybrid_clock.cc:648] HybridClock initialized: now 1790328290640186 us; error 81 us; skew 500 ppm
I20260925 09:24:50.640635 21104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:50.643252 21104 webserver.cc:460] Webserver started at http://127.3.82.62:40391/ using document root <none> and password file <none>
I20260925 09:24:50.643996 21104 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:50.644091 21104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:50.644369 21104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:50.647473 21104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "7d4395f7b04447da9c4f7fb82534b436"
format_stamp: "Formatted at 2026-09-25 09:24:50 on dist-test-slave-4wm5"
server_key: "f2c770e0b06fa5b0e7efc0e77c7a8ee8"
server_key_iv: "8a86b0f4476c730e7f44cf5d522b0661"
server_key_version: "encryptionkey@0"
I20260925 09:24:50.648283 21104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "7d4395f7b04447da9c4f7fb82534b436"
format_stamp: "Formatted at 2026-09-25 09:24:50 on dist-test-slave-4wm5"
server_key: "f2c770e0b06fa5b0e7efc0e77c7a8ee8"
server_key_iv: "8a86b0f4476c730e7f44cf5d522b0661"
server_key_version: "encryptionkey@0"
I20260925 09:24:50.653966 21104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:24:50.657778 21118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:50.659571 21104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:24:50.659760 21104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "7d4395f7b04447da9c4f7fb82534b436"
format_stamp: "Formatted at 2026-09-25 09:24:50 on dist-test-slave-4wm5"
server_key: "f2c770e0b06fa5b0e7efc0e77c7a8ee8"
server_key_iv: "8a86b0f4476c730e7f44cf5d522b0661"
server_key_version: "encryptionkey@0"
I20260925 09:24:50.659970 21104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:50.694443 21104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:50.695348 21104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:50.695608 21104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:50.716501 21104 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37531
I20260925 09:24:50.716512 21169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37531 every 8 connection(s)
I20260925 09:24:50.718036 21104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:24:50.722039 21170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:50.724324 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21104
I20260925 09:24:50.724527 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:24:50.725164 3400 external_mini_cluster.cc:1428] Setting key d8ed5aca9a458f9acdc5eacd5650a4c2
I20260925 09:24:50.735023 21170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436: Bootstrap starting.
I20260925 09:24:50.739751 21170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:50.741045 21170 log.cc:824] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:50.741806 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:50.727552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.3.82.62:37531)
0925 09:24:50.728712 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:50.728725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:50.728769 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:24:50.730721 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:50.730748 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:50.731305 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:50.731695 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:50.732439 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:50.734030 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:50.736104 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:50.737137 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:50.737293 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:50.738809 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:50.738817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:50.738833 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:50.739501 (+ 668us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:50.739605 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:50.739632 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:50.739748 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:50.740126 (+ 378us) server_negotiation.cc:299] Negotiation successful
0925 09:24:50.740532 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":457,"threads_started":1}
I20260925 09:24:50.744510 21170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436: No bootstrap required, opened a new log
I20260925 09:24:50.752501 21170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4395f7b04447da9c4f7fb82534b436" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37531 } }
I20260925 09:24:50.752980 21170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:50.753106 21170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4395f7b04447da9c4f7fb82534b436, State: Initialized, Role: FOLLOWER
I20260925 09:24:50.753767 21170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4395f7b04447da9c4f7fb82534b436" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37531 } }
I20260925 09:24:50.754113 21170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:24:50.754267 21170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:24:50.754456 21170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:50.756721 21170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4395f7b04447da9c4f7fb82534b436" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37531 } }
I20260925 09:24:50.757367 21170 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4395f7b04447da9c4f7fb82534b436; no voters:
I20260925 09:24:50.758165 21170 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:24:50.758268 21175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:50.759130 21175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [term 1 LEADER]: Becoming Leader. State: Replica: 7d4395f7b04447da9c4f7fb82534b436, State: Running, Role: LEADER
I20260925 09:24:50.759699 21175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4395f7b04447da9c4f7fb82534b436" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37531 } }
I20260925 09:24:50.760474 21170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:24:50.763025 21176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d4395f7b04447da9c4f7fb82534b436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4395f7b04447da9c4f7fb82534b436" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37531 } } }
I20260925 09:24:50.762899 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4395f7b04447da9c4f7fb82534b436. Latest consensus state: current_term: 1 leader_uuid: "7d4395f7b04447da9c4f7fb82534b436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4395f7b04447da9c4f7fb82534b436" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37531 } } }
I20260925 09:24:50.763419 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:50.763401 21176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:50.764362 21182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:24:50.777036 21182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:24:50.791103 21182 catalog_manager.cc:1347] Generated new cluster ID: 3fe28a52ee864ba898a53341f3d7d155
I20260925 09:24:50.791328 21182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:24:50.800643 21182 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:24:50.802309 21182 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:24:50.811548 21182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436: Generated new TSK 0
I20260925 09:24:50.812546 21182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:24:50.818143 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37531
--builtin_ntp_servers=127.3.82.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:24:51.211584 21194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:51.211992 21194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:51.212097 21194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:51.219555 21194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:51.219681 21194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:51.219792 21194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:24:51.229396 21194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:51.230732 21194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:51.232697 21194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:51.247380 21200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:51.247411 21199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:51.247449 21202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:51.248129 21194 server_base.cc:1034] running on GCE node
I20260925 09:24:51.248908 21194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:51.249982 21194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:51.251258 21194 hybrid_clock.cc:648] HybridClock initialized: now 1790328291251206 us; error 44 us; skew 500 ppm
I20260925 09:24:51.251634 21194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:51.254382 21194 webserver.cc:460] Webserver started at http://127.3.82.1:34511/ using document root <none> and password file <none>
I20260925 09:24:51.255141 21194 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:51.255277 21194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:51.255620 21194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:51.258715 21194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "e0a5e1047ac843748dc277019797c066"
format_stamp: "Formatted at 2026-09-25 09:24:51 on dist-test-slave-4wm5"
server_key: "a70cc937a5e3970ebdba806352cb9cfe"
server_key_iv: "29926b020038712f214228c0a059f6d9"
server_key_version: "encryptionkey@0"
I20260925 09:24:51.259539 21194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "e0a5e1047ac843748dc277019797c066"
format_stamp: "Formatted at 2026-09-25 09:24:51 on dist-test-slave-4wm5"
server_key: "a70cc937a5e3970ebdba806352cb9cfe"
server_key_iv: "29926b020038712f214228c0a059f6d9"
server_key_version: "encryptionkey@0"
I20260925 09:24:51.265600 21194 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:24:51.270102 21208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:51.272076 21194 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:24:51.272334 21194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "e0a5e1047ac843748dc277019797c066"
format_stamp: "Formatted at 2026-09-25 09:24:51 on dist-test-slave-4wm5"
server_key: "a70cc937a5e3970ebdba806352cb9cfe"
server_key_iv: "29926b020038712f214228c0a059f6d9"
server_key_version: "encryptionkey@0"
I20260925 09:24:51.272643 21194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:51.293057 21194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:51.294103 21194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:51.294426 21194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:51.295511 21194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:51.297169 21194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:51.297312 21194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:51.297458 21194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:51.297559 21194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:51.337590 21194 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:40287
I20260925 09:24:51.337697 21320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:40287 every 8 connection(s)
I20260925 09:24:51.339293 21194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:24:51.344033 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21194
I20260925 09:24:51.344290 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:24:51.344954 3400 external_mini_cluster.cc:1428] Setting key 8d26e31d8fc9bd249790aa4978e1b6d4
I20260925 09:24:51.348439 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37531
--builtin_ntp_servers=127.3.82.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:24:51.356089 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:51.342249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:42099 (local address 127.3.82.62:37531)
0925 09:24:51.342738 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:51.342745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:51.344514 (+ 1769us) server_negotiation.cc:407] Connection header received
0925 09:24:51.347230 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:51.347235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:51.347340 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:51.347460 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:51.348855 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.350291 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.352154 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.352985 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.353155 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:51.354863 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:51.354873 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:51.354881 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:51.354913 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:51.354972 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:51.354993 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:51.355106 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:51.355428 (+ 322us) server_negotiation.cc:299] Negotiation successful
0925 09:24:51.355564 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":232,"threads_started":1}
I20260925 09:24:51.357230 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:51.342886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37531 (local address 127.3.82.1:42099)
0925 09:24:51.344306 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:51.344370 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:51.346752 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:51.347741 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:51.347751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:51.347933 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:51.348634 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:51.348666 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.350519 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.350523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:51.351972 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:51.351986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.353291 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.353297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:51.353567 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:51.354467 (+ 900us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:24:51.354470 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:51.354516 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:24:51.354610 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:51.355145 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:51.355156 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:51.355164 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 09:24:51.355385 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 09:24:51.355781 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":324,"threads_started":1}
I20260925 09:24:51.359354 21321 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37531
I20260925 09:24:51.359735 21321 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:51.360925 21321 heartbeater.cc:511] Master 127.3.82.62:37531 requested a full tablet report, sending...
I20260925 09:24:51.363831 21135 ts_manager.cc:194] Registered new tserver with Master: e0a5e1047ac843748dc277019797c066 (127.3.82.1:40287)
I20260925 09:24:51.366336 21135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.1:42099
W20260925 09:24:51.711213 21325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:51.711531 21325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:51.711621 21325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:51.719655 21325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:51.719827 21325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:51.720039 21325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:24:51.729890 21325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:51.731267 21325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:51.733150 21325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:51.747556 21330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:51.747534 21333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:51.747534 21331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:51.748569 21325 server_base.cc:1034] running on GCE node
I20260925 09:24:51.749445 21325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:51.750775 21325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:51.752023 21325 hybrid_clock.cc:648] HybridClock initialized: now 1790328291751984 us; error 40 us; skew 500 ppm
I20260925 09:24:51.752425 21325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:51.755165 21325 webserver.cc:460] Webserver started at http://127.3.82.2:32863/ using document root <none> and password file <none>
I20260925 09:24:51.755967 21325 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:51.756062 21325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:51.756433 21325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:51.759625 21325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "d64b4782d9954a899f131d4f1b4ab97f"
format_stamp: "Formatted at 2026-09-25 09:24:51 on dist-test-slave-4wm5"
server_key: "341a30d81f841484016cdeb5fb9b4f27"
server_key_iv: "d6236f313c390717650c7ef2f446729b"
server_key_version: "encryptionkey@0"
I20260925 09:24:51.760648 21325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "d64b4782d9954a899f131d4f1b4ab97f"
format_stamp: "Formatted at 2026-09-25 09:24:51 on dist-test-slave-4wm5"
server_key: "341a30d81f841484016cdeb5fb9b4f27"
server_key_iv: "d6236f313c390717650c7ef2f446729b"
server_key_version: "encryptionkey@0"
I20260925 09:24:51.766753 21325 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:24:51.771121 21339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:51.772944 21325 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:24:51.773183 21325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "d64b4782d9954a899f131d4f1b4ab97f"
format_stamp: "Formatted at 2026-09-25 09:24:51 on dist-test-slave-4wm5"
server_key: "341a30d81f841484016cdeb5fb9b4f27"
server_key_iv: "d6236f313c390717650c7ef2f446729b"
server_key_version: "encryptionkey@0"
I20260925 09:24:51.773463 21325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:51.820715 21325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:51.821884 21325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:51.822223 21325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:51.823369 21325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:51.825176 21325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:51.825317 21325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:51.825467 21325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:51.825558 21325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:51.864143 21325 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:44759
I20260925 09:24:51.864238 21451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:44759 every 8 connection(s)
I20260925 09:24:51.865595 21325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:24:51.873371 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21325
I20260925 09:24:51.873541 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:24:51.874063 3400 external_mini_cluster.cc:1428] Setting key 1e301af235ae3eae2b46f49fd1b1650d
I20260925 09:24:51.877213 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37531
--builtin_ntp_servers=127.3.82.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:24:51.880597 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:51.868512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:43307 (local address 127.3.82.62:37531)
0925 09:24:51.869064 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:51.869071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:51.870614 (+ 1543us) server_negotiation.cc:407] Connection header received
0925 09:24:51.872692 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:51.872697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:51.872775 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:51.872886 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:51.874462 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.875445 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.877246 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.877907 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.878052 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:51.879692 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:51.879698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:51.879703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:51.879732 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:51.879774 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:51.879786 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:51.879888 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:51.880124 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 09:24:51.880236 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":235,"threads_started":1}
I20260925 09:24:51.881758 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:51.869223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37531 (local address 127.3.82.2:43307)
0925 09:24:51.870413 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:51.870457 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:51.872376 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:51.873218 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:51.873228 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:51.873335 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:51.874250 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:51.874272 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.875626 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.875634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:51.877082 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:51.877095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:51.878157 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:51.878167 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:51.878442 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:51.879314 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:24:51.879317 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:51.879363 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:24:51.879471 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:51.879887 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:51.879899 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:51.879910 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:24:51.880114 (+ 204us) client_negotiation.cc:240] Negotiation successful
0925 09:24:51.880600 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":313,"threads_started":1}
I20260925 09:24:51.883674 21452 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37531
I20260925 09:24:51.884100 21452 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:51.884979 21452 heartbeater.cc:511] Master 127.3.82.62:37531 requested a full tablet report, sending...
I20260925 09:24:51.886732 21135 ts_manager.cc:194] Registered new tserver with Master: d64b4782d9954a899f131d4f1b4ab97f (127.3.82.2:44759)
I20260925 09:24:51.887627 21135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.2:43307
W20260925 09:24:52.235189 21456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:52.235545 21456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:52.235718 21456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:52.244334 21456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:52.244508 21456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:52.244699 21456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:24:52.254703 21456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:52.256181 21456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:52.258100 21456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:52.273391 21462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:52.273404 21461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:52.273432 21464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:52.274155 21456 server_base.cc:1034] running on GCE node
I20260925 09:24:52.275010 21456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:52.276072 21456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:52.277333 21456 hybrid_clock.cc:648] HybridClock initialized: now 1790328292277266 us; error 55 us; skew 500 ppm
I20260925 09:24:52.277748 21456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:52.280692 21456 webserver.cc:460] Webserver started at http://127.3.82.3:43599/ using document root <none> and password file <none>
I20260925 09:24:52.281514 21456 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:52.281618 21456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:52.282227 21456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:52.285919 21456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "4ed1089620924aec958d453d0bdff496"
format_stamp: "Formatted at 2026-09-25 09:24:52 on dist-test-slave-4wm5"
server_key: "e48c9e7fb06047341170d1fd38711f10"
server_key_iv: "7b82803977c969c92615256b6e27cd92"
server_key_version: "encryptionkey@0"
I20260925 09:24:52.286863 21456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "4ed1089620924aec958d453d0bdff496"
format_stamp: "Formatted at 2026-09-25 09:24:52 on dist-test-slave-4wm5"
server_key: "e48c9e7fb06047341170d1fd38711f10"
server_key_iv: "7b82803977c969c92615256b6e27cd92"
server_key_version: "encryptionkey@0"
I20260925 09:24:52.292676 21456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:24:52.296975 21470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:52.298647 21456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:24:52.298813 21456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "4ed1089620924aec958d453d0bdff496"
format_stamp: "Formatted at 2026-09-25 09:24:52 on dist-test-slave-4wm5"
server_key: "e48c9e7fb06047341170d1fd38711f10"
server_key_iv: "7b82803977c969c92615256b6e27cd92"
server_key_version: "encryptionkey@0"
I20260925 09:24:52.299126 21456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:52.317428 21456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:52.318454 21456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:52.318766 21456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:52.319873 21456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:52.321555 21456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:52.321648 21456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:52.321794 21456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:52.321890 21456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260925 09:24:52.361366 21456 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:40581
I20260925 09:24:52.361470 21582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:40581 every 8 connection(s)
I20260925 09:24:52.363057 21456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:24:52.366528 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21456
I20260925 09:24:52.366725 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:24:52.367216 3400 external_mini_cluster.cc:1428] Setting key cea6b4559a4a6d1e3b5afbd7125b353a
I20260925 09:24:52.370999 21321 heartbeater.cc:503] Master 127.3.82.62:37531 was elected leader, sending a full tablet report...
I20260925 09:24:52.378592 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.366090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:51379 (local address 127.3.82.62:37531)
0925 09:24:52.366346 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.366354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.368304 (+ 1950us) server_negotiation.cc:407] Connection header received
0925 09:24:52.370521 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.370525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.370605 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.370704 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:52.372343 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.373393 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.375170 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.375653 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.375775 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.377552 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:52.377558 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:52.377561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:52.377583 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:52.377616 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:52.377629 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:52.377738 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.378072 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.378210 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 09:24:52.379868 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.366853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37531 (local address 127.3.82.3:51379)
0925 09:24:52.368138 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.368187 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:52.370237 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:52.371049 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:52.371059 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:52.371199 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:52.372149 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.372170 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.373552 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.373556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.375005 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.375019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.375966 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.375973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.376253 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.377092 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:24:52.377101 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:52.377191 (+ 90us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:24:52.377300 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:52.377756 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:52.377768 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:52.377780 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 09:24:52.378062 (+ 282us) client_negotiation.cc:240] Negotiation successful
0925 09:24:52.378556 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":389,"threads_started":1}
I20260925 09:24:52.381736 21583 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37531
I20260925 09:24:52.382148 21583 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:52.383230 21583 heartbeater.cc:511] Master 127.3.82.62:37531 requested a full tablet report, sending...
I20260925 09:24:52.384865 21135 ts_manager.cc:194] Registered new tserver with Master: 4ed1089620924aec958d453d0bdff496 (127.3.82.3:40581)
I20260925 09:24:52.386046 21135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.3:51379
I20260925 09:24:52.396571 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:24:52.413123 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.405420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.3.82.62:37531)
0925 09:24:52.405703 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.405712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.406112 (+ 400us) server_negotiation.cc:407] Connection header received
0925 09:24:52.406380 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.406386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.406461 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.406584 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:52.407155 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.407960 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.409758 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.410632 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.410817 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.412196 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:52.412204 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:52.412211 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:52.412240 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:52.412301 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:52.412331 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:52.412449 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.412614 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.412727 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260925 09:24:52.417426 21135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:24:52.424566 21135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:24:52.454250 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.438470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:44759 (local address 127.0.0.1:45964)
0925 09:24:52.441568 (+ 3098us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.441626 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:52.441929 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:52.443270 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:52.443283 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:52.443339 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:52.443984 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.444003 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.446790 (+ 2787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.446800 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.450666 (+ 3866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.450684 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.453142 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.453148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.453399 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.453471 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 09:24:52.453625 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 09:24:52.453868 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2652,"thread_start_us":12547,"threads_started":1}
I20260925 09:24:52.454268 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.438711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.3.82.2:44759)
0925 09:24:52.440884 (+ 2173us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.440902 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.442209 (+ 1307us) server_negotiation.cc:407] Connection header received
0925 09:24:52.442370 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.442381 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.442706 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.442917 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:52.444302 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.446529 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.450978 (+ 4449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.452899 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.453079 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.453364 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.453685 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.453894 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1999,"thread_start_us":1859,"threads_started":1}
I20260925 09:24:52.457918 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.439095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43818 (local address 127.3.82.1:40287)
0925 09:24:52.440430 (+ 1335us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.440442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.444180 (+ 3738us) server_negotiation.cc:407] Connection header received
0925 09:24:52.444574 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.444586 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.444935 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.445152 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:52.448393 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.450574 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.454576 (+ 4002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.456210 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.456365 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.456643 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.457284 (+ 641us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.457495 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":1841,"threads_started":1}
I20260925 09:24:52.458415 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.438603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43818)
0925 09:24:52.444010 (+ 5407us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.444038 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:52.444339 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:52.445587 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:52.445594 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:52.445631 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:52.448081 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.448105 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.451982 (+ 3877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.451996 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.454333 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.454349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.456770 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.456777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.457025 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.457080 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 09:24:52.457909 (+ 829us) client_negotiation.cc:240] Negotiation successful
0925 09:24:52.458066 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5217,"mutex_wait_us":33,"thread_start_us":2525,"threads_started":1}
I20260925 09:24:52.458843 21387 tablet_service.cc:1467] Processing CreateTablet for tablet b98d1e974137472f9aa0c3002500301c (DEFAULT_TABLE table=test-table [id=58bb9ca383b446afa479469b422eb09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:52.459882 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.440288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:40581 (local address 127.0.0.1:57214)
0925 09:24:52.450777 (+ 10489us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.450801 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:52.450934 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:52.452488 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:52.452494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:52.452528 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:52.452942 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.452956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.455002 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.455007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.457079 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.457092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.459063 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.459070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.459241 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.459285 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 09:24:52.459398 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 09:24:52.459532 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10301,"mutex_wait_us":27,"thread_start_us":3360,"threads_started":1}
I20260925 09:24:52.460156 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.440969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.3.82.3:40581)
0925 09:24:52.451771 (+ 10802us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.451780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.451822 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:24:52.451928 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.451937 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.452237 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.452487 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:52.453200 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.454829 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.457308 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.458853 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.459020 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.459295 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.459441 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.459604 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10617,"thread_start_us":10439,"threads_started":1}
I20260925 09:24:52.462286 21256 tablet_service.cc:1467] Processing CreateTablet for tablet b98d1e974137472f9aa0c3002500301c (DEFAULT_TABLE table=test-table [id=58bb9ca383b446afa479469b422eb09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:52.463927 21256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98d1e974137472f9aa0c3002500301c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:52.463439 21518 tablet_service.cc:1467] Processing CreateTablet for tablet b98d1e974137472f9aa0c3002500301c (DEFAULT_TABLE table=test-table [id=58bb9ca383b446afa479469b422eb09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:52.464787 21387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98d1e974137472f9aa0c3002500301c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:52.464856 21518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98d1e974137472f9aa0c3002500301c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:52.477360 21601 tablet_bootstrap.cc:492] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Bootstrap starting.
I20260925 09:24:52.479657 21602 tablet_bootstrap.cc:492] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Bootstrap starting.
I20260925 09:24:52.482694 21601 tablet_bootstrap.cc:654] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:52.483536 21602 tablet_bootstrap.cc:654] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:52.484118 21601 log.cc:824] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:52.484788 21602 log.cc:824] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:52.486953 21605 tablet_bootstrap.cc:492] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Bootstrap starting.
I20260925 09:24:52.487596 21601 tablet_bootstrap.cc:492] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: No bootstrap required, opened a new log
I20260925 09:24:52.487887 21601 ts_tablet_manager.cc:1397] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260925 09:24:52.489665 21602 tablet_bootstrap.cc:492] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: No bootstrap required, opened a new log
I20260925 09:24:52.489985 21602 ts_tablet_manager.cc:1397] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260925 09:24:52.493053 21605 tablet_bootstrap.cc:654] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:52.494216 21605 log.cc:824] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:52.497090 21605 tablet_bootstrap.cc:492] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: No bootstrap required, opened a new log
I20260925 09:24:52.497390 21605 ts_tablet_manager.cc:1397] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260925 09:24:52.498811 21601 raft_consensus.cc:348] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.499295 21601 raft_consensus.cc:374] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:52.499455 21601 raft_consensus.cc:729] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64b4782d9954a899f131d4f1b4ab97f, State: Initialized, Role: FOLLOWER
I20260925 09:24:52.500133 21601 consensus_queue.cc:260] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.500761 21602 raft_consensus.cc:348] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.501268 21602 raft_consensus.cc:374] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:52.501332 21452 heartbeater.cc:503] Master 127.3.82.62:37531 was elected leader, sending a full tablet report...
I20260925 09:24:52.501386 21602 raft_consensus.cc:729] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed1089620924aec958d453d0bdff496, State: Initialized, Role: FOLLOWER
I20260925 09:24:52.502082 21602 consensus_queue.cc:260] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.503415 21583 heartbeater.cc:503] Master 127.3.82.62:37531 was elected leader, sending a full tablet report...
I20260925 09:24:52.503901 21601 ts_tablet_manager.cc:1428] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260925 09:24:52.504369 21602 ts_tablet_manager.cc:1428] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260925 09:24:52.509145 21605 raft_consensus.cc:348] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.509555 21605 raft_consensus.cc:374] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:52.509809 21605 raft_consensus.cc:729] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a5e1047ac843748dc277019797c066, State: Initialized, Role: FOLLOWER
I20260925 09:24:52.510617 21605 consensus_queue.cc:260] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.512518 21605 ts_tablet_manager.cc:1428] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260925 09:24:52.526021 21609 raft_consensus.cc:482] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:24:52.526340 21609 raft_consensus.cc:504] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.528126 21609 leader_election.cc:288] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d64b4782d9954a899f131d4f1b4ab97f (127.3.82.2:44759), 4ed1089620924aec958d453d0bdff496 (127.3.82.3:40581)
I20260925 09:24:52.538121 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.528659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:55365 (local address 127.3.82.2:44759)
0925 09:24:52.528859 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.528865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.529618 (+ 753us) server_negotiation.cc:407] Connection header received
0925 09:24:52.529827 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.529850 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.529947 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.530045 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:52.530923 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.532095 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.535451 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.536713 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.536866 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.536982 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.537641 (+ 659us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.537799 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:24:52.538149 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.528660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:48953 (local address 127.3.82.3:40581)
0925 09:24:52.528859 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.528865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.529617 (+ 752us) server_negotiation.cc:407] Connection header received
0925 09:24:52.529940 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.529947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.530037 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.530123 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:52.530981 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.532095 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.535460 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.536718 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.536951 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.537070 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.537640 (+ 570us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.537799 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:24:52.538249 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.528551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:44759 (local address 127.3.82.1:55365)
0925 09:24:52.529474 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.529495 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:52.529653 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:52.530074 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:52.530077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:52.530131 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:52.530711 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.530720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.532289 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.532295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.535144 (+ 2849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.535166 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.537055 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.537064 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.537312 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.537401 (+ 89us) client_negotiation.cc:769] Sending connection context
0925 09:24:52.537527 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 09:24:52.537707 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":232,"thread_start_us":547,"threads_started":1}
I20260925 09:24:52.538275 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.528524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:40581 (local address 127.3.82.1:48953)
0925 09:24:52.529474 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.529495 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:52.529695 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:52.530147 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:52.530150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:52.530172 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:52.530707 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.530720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.532289 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.532295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.535144 (+ 2849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:52.535166 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.537069 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.537074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:52.537304 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.537401 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 09:24:52.537522 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 09:24:52.537706 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":265,"spinlock_wait_cycles":35072,"thread_start_us":529,"threads_started":1}
I20260925 09:24:52.540021 21407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98d1e974137472f9aa0c3002500301c" candidate_uuid: "e0a5e1047ac843748dc277019797c066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64b4782d9954a899f131d4f1b4ab97f" is_pre_election: true
I20260925 09:24:52.540058 21538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98d1e974137472f9aa0c3002500301c" candidate_uuid: "e0a5e1047ac843748dc277019797c066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed1089620924aec958d453d0bdff496" is_pre_election: true
I20260925 09:24:52.540706 21407 raft_consensus.cc:2454] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a5e1047ac843748dc277019797c066 in term 0.
I20260925 09:24:52.540699 21538 raft_consensus.cc:2454] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a5e1047ac843748dc277019797c066 in term 0.
I20260925 09:24:52.541642 21210 leader_election.cc:302] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ed1089620924aec958d453d0bdff496, d64b4782d9954a899f131d4f1b4ab97f, e0a5e1047ac843748dc277019797c066; no voters:
I20260925 09:24:52.542275 21609 raft_consensus.cc:2781] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:24:52.542443 21609 raft_consensus.cc:482] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:24:52.542580 21609 raft_consensus.cc:3037] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:52.545099 21609 raft_consensus.cc:504] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.545858 21609 leader_election.cc:288] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [CANDIDATE]: Term 1 election: Requested vote from peers d64b4782d9954a899f131d4f1b4ab97f (127.3.82.2:44759), 4ed1089620924aec958d453d0bdff496 (127.3.82.3:40581)
I20260925 09:24:52.546514 21407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98d1e974137472f9aa0c3002500301c" candidate_uuid: "e0a5e1047ac843748dc277019797c066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64b4782d9954a899f131d4f1b4ab97f"
I20260925 09:24:52.546543 21538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98d1e974137472f9aa0c3002500301c" candidate_uuid: "e0a5e1047ac843748dc277019797c066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed1089620924aec958d453d0bdff496"
I20260925 09:24:52.546742 21407 raft_consensus.cc:3037] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:52.546790 21538 raft_consensus.cc:3037] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:52.549192 21538 raft_consensus.cc:2454] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a5e1047ac843748dc277019797c066 in term 1.
I20260925 09:24:52.549296 21407 raft_consensus.cc:2454] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a5e1047ac843748dc277019797c066 in term 1.
I20260925 09:24:52.549742 21210 leader_election.cc:302] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed1089620924aec958d453d0bdff496, e0a5e1047ac843748dc277019797c066; no voters:
I20260925 09:24:52.550127 21609 raft_consensus.cc:2781] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:52.550627 21609 raft_consensus.cc:686] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 1 LEADER]: Becoming Leader. State: Replica: e0a5e1047ac843748dc277019797c066, State: Running, Role: LEADER
I20260925 09:24:52.551102 21609 consensus_queue.cc:237] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } }
I20260925 09:24:52.556303 21133 catalog_manager.cc:5495] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a5e1047ac843748dc277019797c066 (127.3.82.1). New cstate: current_term: 1 leader_uuid: "e0a5e1047ac843748dc277019797c066" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0a5e1047ac843748dc277019797c066" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 } health_report { overall_health: UNKNOWN } } }
I20260925 09:24:52.580262 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.574350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.3.82.1:40287)
0925 09:24:52.574541 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.574546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.574561 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:24:52.574648 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.574652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.574736 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.574863 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:24:52.575236 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.575977 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.577784 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.578450 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.578568 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.579125 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:24:52.579687 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:24:52.579799 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.579989 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.580058 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:24:52.591140 21407 raft_consensus.cc:1261] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 1 FOLLOWER]: Refusing update from remote peer e0a5e1047ac843748dc277019797c066: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:52.591137 21538 raft_consensus.cc:1261] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 1 FOLLOWER]: Refusing update from remote peer e0a5e1047ac843748dc277019797c066: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:52.592337 21612 consensus_queue.cc:1035] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d64b4782d9954a899f131d4f1b4ab97f" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 44759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:24:52.592617 21322 tablet.cc:2369] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:24:52.592799 21609 consensus_queue.cc:1035] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed1089620924aec958d453d0bdff496" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:52.608606 21617 mvcc.cc:205] Tried to move back new op lower bound from 7333184686440546304 to 7333184686293352448. Current Snapshot: MvccSnapshot[applied={T|T < 7333184686440546304}]
I20260925 09:24:52.624717 21135 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:52.625097 21135 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:52.629190 21135 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d4395f7b04447da9c4f7fb82534b436: Sending DeleteTablet for 3 replicas of tablet b98d1e974137472f9aa0c3002500301c
I20260925 09:24:52.630532 21387 tablet_service.cc:1514] Processing DeleteTablet for tablet b98d1e974137472f9aa0c3002500301c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:52 UTC) from {username='slave'} at 127.0.0.1:45964
I20260925 09:24:52.630577 21518 tablet_service.cc:1514] Processing DeleteTablet for tablet b98d1e974137472f9aa0c3002500301c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:52 UTC) from {username='slave'} at 127.0.0.1:57214
I20260925 09:24:52.631299 21256 tablet_service.cc:1514] Processing DeleteTablet for tablet b98d1e974137472f9aa0c3002500301c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:52 UTC) from {username='slave'} at 127.0.0.1:43818
I20260925 09:24:52.632100 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21194
I20260925 09:24:52.646503 21627 tablet_replica.cc:330] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: stopping tablet replica
I20260925 09:24:52.650861 21626 tablet_replica.cc:330] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: stopping tablet replica
I20260925 09:24:52.651711 21627 raft_consensus.cc:2229] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:24:52.652016 21626 raft_consensus.cc:2229] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:52.652534 21627 raft_consensus.cc:2258] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:52.652618 21626 raft_consensus.cc:2258] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:52.653399 21625 tablet_replica.cc:330] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: stopping tablet replica
I20260925 09:24:52.654202 21625 raft_consensus.cc:2229] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:52.655388 21626 ts_tablet_manager.cc:1905] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:52.655643 21627 ts_tablet_manager.cc:1905] T b98d1e974137472f9aa0c3002500301c P e0a5e1047ac843748dc277019797c066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:52.660406 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.635384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.3.82.1:40287)
0925 09:24:52.635637 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.635646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.644185 (+ 8539us) server_negotiation.cc:407] Connection header received
0925 09:24:52.644813 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.644822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.644986 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.645080 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:52.646847 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.648458 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.651033 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.651764 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.651899 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.658441 (+ 6542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:52.658456 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:52.658476 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:52.659320 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:52.659393 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:52.659421 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:52.659532 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.659800 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.659938 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 09:24:52.660916 21625 raft_consensus.cc:2258] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:52.662165 21316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:52.663085 21626 ts_tablet_manager.cc:1918] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:52.663312 21626 log.cc:1196] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/b98d1e974137472f9aa0c3002500301c
I20260925 09:24:52.664106 21626 ts_tablet_manager.cc:1939] T b98d1e974137472f9aa0c3002500301c P d64b4782d9954a899f131d4f1b4ab97f: Deleting consensus metadata
I20260925 09:24:52.664561 21625 ts_tablet_manager.cc:1905] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:52.666724 21122 catalog_manager.cc:4841] TS d64b4782d9954a899f131d4f1b4ab97f (127.3.82.2:44759): tablet b98d1e974137472f9aa0c3002500301c (table test-table [id=58bb9ca383b446afa479469b422eb09f]) successfully deleted
I20260925 09:24:52.671639 21625 ts_tablet_manager.cc:1918] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:52.671833 21625 log.cc:1196] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/b98d1e974137472f9aa0c3002500301c
I20260925 09:24:52.672330 21625 ts_tablet_manager.cc:1939] T b98d1e974137472f9aa0c3002500301c P 4ed1089620924aec958d453d0bdff496: Deleting consensus metadata
I20260925 09:24:52.674134 21120 catalog_manager.cc:4841] TS 4ed1089620924aec958d453d0bdff496 (127.3.82.3:40581): tablet b98d1e974137472f9aa0c3002500301c (table test-table [id=58bb9ca383b446afa479469b422eb09f]) successfully deleted
I20260925 09:24:52.768285 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21194
W20260925 09:24:52.787655 21121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.82.1:40287 (error 108)
I20260925 09:24:52.788054 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21325
I20260925 09:24:52.789963 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.789410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43842)
0925 09:24:52.789605 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.789711 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260925 09:24:52.790249 21121 catalog_manager.cc:4584] TS e0a5e1047ac843748dc277019797c066 (127.3.82.1:40287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b98d1e974137472f9aa0c3002500301c: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
I20260925 09:24:52.796198 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.788976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.3.82.2:44759)
0925 09:24:52.789175 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.789180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.789320 (+ 140us) server_negotiation.cc:407] Connection header received
0925 09:24:52.789586 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.789597 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.789683 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.789765 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:52.790328 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.791145 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.792560 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.793017 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.793170 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.794587 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:52.794596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:52.794612 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:52.795371 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:52.795424 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:52.795445 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:52.795586 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.795734 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.795867 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 09:24:52.796950 21447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:52.844183 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.843619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43848)
0925 09:24:52.843912 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.844012 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260925 09:24:52.845417 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.845058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43862)
0925 09:24:52.845206 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.845278 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 09:24:52.895279 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21325
I20260925 09:24:52.906461 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.905775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43866)
0925 09:24:52.906083 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.906207 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260925 09:24:52.908071 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.907558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43870)
0925 09:24:52.907758 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.907866 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 09:24:52.914002 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21456
I20260925 09:24:52.921312 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.914846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57226 (local address 127.3.82.3:40581)
0925 09:24:52.915118 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:52.915124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:52.915143 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:24:52.915297 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:52.915303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:52.915403 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:52.915525 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:52.915953 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.916706 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.918234 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:52.918644 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:52.918782 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:52.919935 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:52.919945 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:52.919960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:52.920541 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:52.920592 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:52.920611 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:52.920741 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:52.920878 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 09:24:52.920999 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260925 09:24:52.922380 21578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:52.989094 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.988569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43876)
0925 09:24:52.988815 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.988940 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260925 09:24:52.990250 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:52.989784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40287 (local address 127.0.0.1:43882)
0925 09:24:52.990051 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:52.990137 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.1:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260925 09:24:53.014925 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21456
I20260925 09:24:53.032109 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21104
I20260925 09:24:53.033245 21165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:53.147188 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21104
2026-09-25T09:24:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2983 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:24:53 dist-test-slave-4wm5 krb5kdc[21636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:24:53 dist-test-slave-4wm5 krb5kdc[21636](info): set up 2 sockets
Sep 25 09:24:53 dist-test-slave-4wm5 krb5kdc[21636](info): commencing operation
krb5kdc: starting...
W20260925 09:24:54.290447 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:24:54 dist-test-slave-4wm5 krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:24:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:24:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:24:54.566754 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:35127
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:35127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:54.934170 21649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:54.934494 21649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:54.934721 21649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:54.942653 21649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:24:54.942782 21649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:54.942873 21649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:54.942967 21649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:24:54.943087 21649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:24:54.954311 21649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:35127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:54.955734 21649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:54.957165 21649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:54.970508 21654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:54.970598 21655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:54.970785 21657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:54.971692 21649 server_base.cc:1034] running on GCE node
I20260925 09:24:54.972615 21649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:54.974323 21649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:54.975605 21649 hybrid_clock.cc:648] HybridClock initialized: now 1790328294975541 us; error 53 us; skew 500 ppm
I20260925 09:24:54.976099 21649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:54 dist-test-slave-4wm5 krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328294, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:54.984004 21649 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:24:54.985756 21649 webserver.cc:460] Webserver started at http://127.3.82.62:41469/ using document root <none> and password file <none>
I20260925 09:24:54.986572 21649 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:54.986685 21649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:54.986938 21649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:54.990167 21649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "2b92e80846df46baa4d23f993383a9b0"
format_stamp: "Formatted at 2026-09-25 09:24:54 on dist-test-slave-4wm5"
server_key: "f15d7f269015b204f5af096e208e9827"
server_key_iv: "2ad02bdfa05357d88ba8e903029cd698"
server_key_version: "encryptionkey@0"
I20260925 09:24:54.991053 21649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "2b92e80846df46baa4d23f993383a9b0"
format_stamp: "Formatted at 2026-09-25 09:24:54 on dist-test-slave-4wm5"
server_key: "f15d7f269015b204f5af096e208e9827"
server_key_iv: "2ad02bdfa05357d88ba8e903029cd698"
server_key_version: "encryptionkey@0"
I20260925 09:24:54.997006 21649 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:24:55.001143 21664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:55.002900 21649 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:24:55.003152 21649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2b92e80846df46baa4d23f993383a9b0"
format_stamp: "Formatted at 2026-09-25 09:24:54 on dist-test-slave-4wm5"
server_key: "f15d7f269015b204f5af096e208e9827"
server_key_iv: "2ad02bdfa05357d88ba8e903029cd698"
server_key_version: "encryptionkey@0"
I20260925 09:24:55.003402 21649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:55.022378 21649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:55.027904 21649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:55.028168 21649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:55.048780 21649 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:35127
I20260925 09:24:55.048918 21715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:35127 every 8 connection(s)
I20260925 09:24:55.050365 21649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:24:55.054455 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21649
I20260925 09:24:55.054471 21716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:55.054626 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:24:55.055075 3400 external_mini_cluster.cc:1428] Setting key db77550cba3f982edf8523440aa4b20d
I20260925 09:24:55.070019 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0: Bootstrap starting.
Sep 25 09:24:55 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:55.073642 21716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:55.075027 21716 log.cc:824] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:55.078516 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0: No bootstrap required, opened a new log
I20260925 09:24:55.081619 21719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:55.056977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.3.82.62:35127)
0925 09:24:55.057957 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:55.057973 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:55.058019 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:24:55.059611 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:55.059637 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:55.060075 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:55.060452 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:55.062370 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:55.063677 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:55.065893 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:55.066751 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:55.066949 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:55.072459 (+ 5510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:55.072470 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:55.072487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:55.072540 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:55.076607 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:55.077414 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:55.077423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:55.077430 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:55.077600 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:55.078094 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:55.078100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:55.078110 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:55.078676 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:55.078974 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:55.079572 (+ 598us) server_negotiation.cc:299] Negotiation successful
0925 09:24:55.080053 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":409,"threads_started":1}
I20260925 09:24:55.088932 21716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b92e80846df46baa4d23f993383a9b0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35127 } }
I20260925 09:24:55.089356 21716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:55.089442 21716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b92e80846df46baa4d23f993383a9b0, State: Initialized, Role: FOLLOWER
I20260925 09:24:55.090171 21716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b92e80846df46baa4d23f993383a9b0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35127 } }
I20260925 09:24:55.090418 21716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:24:55.090528 21716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:24:55.090669 21716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:55.093050 21716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b92e80846df46baa4d23f993383a9b0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35127 } }
I20260925 09:24:55.093562 21716 leader_election.cc:302] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b92e80846df46baa4d23f993383a9b0; no voters:
I20260925 09:24:55.094265 21716 leader_election.cc:288] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:24:55.094432 21721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:55.095115 21721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 2b92e80846df46baa4d23f993383a9b0, State: Running, Role: LEADER
I20260925 09:24:55.095764 21721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b92e80846df46baa4d23f993383a9b0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35127 } }
I20260925 09:24:55.096805 21716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:24:55.099117 21723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b92e80846df46baa4d23f993383a9b0. Latest consensus state: current_term: 1 leader_uuid: "2b92e80846df46baa4d23f993383a9b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b92e80846df46baa4d23f993383a9b0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35127 } } }
I20260925 09:24:55.099771 21723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:55.100374 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b92e80846df46baa4d23f993383a9b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b92e80846df46baa4d23f993383a9b0" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35127 } } }
I20260925 09:24:55.100864 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:55.102114 21730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:24:55.107796 21730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:24:55.118657 21730 catalog_manager.cc:1347] Generated new cluster ID: 1cb228fb1a0b41eba5079487cded3b0e
I20260925 09:24:55.118820 21730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:24:55.149765 21730 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:24:55.151328 21730 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:24:55.162247 21730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0: Generated new TSK 0
I20260925 09:24:55.163305 21730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:24:55.234154 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35127
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:55.601596 21742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:55.602011 21742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:55.602207 21742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:55.610028 21742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:55.610210 21742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:55.610391 21742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:24:55.620836 21742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:55.622699 21742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:55.624191 21742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:55.639884 21750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:55.639912 21747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:55.639776 21748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:55.643972 21742 server_base.cc:1034] running on GCE node
I20260925 09:24:55.644744 21742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:55.646080 21742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:55.647322 21742 hybrid_clock.cc:648] HybridClock initialized: now 1790328295647276 us; error 43 us; skew 500 ppm
I20260925 09:24:55.647670 21742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:55 dist-test-slave-4wm5 krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328295, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:55.651280 21742 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:24:55.652953 21742 webserver.cc:460] Webserver started at http://127.3.82.1:37151/ using document root <none> and password file <none>
I20260925 09:24:55.653782 21742 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:55.653988 21742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:55.654285 21742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:55.657446 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "8fe67345abe44f93ba7e24de87c4c256"
format_stamp: "Formatted at 2026-09-25 09:24:55 on dist-test-slave-4wm5"
server_key: "20a2460d52d7a41d74c069323b27bd63"
server_key_iv: "930eb5b947e4d2e0b095c7ec07c666f9"
server_key_version: "encryptionkey@0"
I20260925 09:24:55.658396 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "8fe67345abe44f93ba7e24de87c4c256"
format_stamp: "Formatted at 2026-09-25 09:24:55 on dist-test-slave-4wm5"
server_key: "20a2460d52d7a41d74c069323b27bd63"
server_key_iv: "930eb5b947e4d2e0b095c7ec07c666f9"
server_key_version: "encryptionkey@0"
I20260925 09:24:55.664433 21742 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:24:55.668705 21757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:55.670549 21742 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260925 09:24:55.670756 21742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "8fe67345abe44f93ba7e24de87c4c256"
format_stamp: "Formatted at 2026-09-25 09:24:55 on dist-test-slave-4wm5"
server_key: "20a2460d52d7a41d74c069323b27bd63"
server_key_iv: "930eb5b947e4d2e0b095c7ec07c666f9"
server_key_version: "encryptionkey@0"
I20260925 09:24:55.670996 21742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:55.687565 21742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:55.692551 21742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:55.692875 21742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:55.694022 21742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:55.695451 21742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:55.695524 21742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:55.695654 21742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:55.695711 21742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:55.732900 21742 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:36175
I20260925 09:24:55.732954 21869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:36175 every 8 connection(s)
I20260925 09:24:55.734496 21742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:24:55.735126 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21742
I20260925 09:24:55.735329 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:24:55.736066 3400 external_mini_cluster.cc:1428] Setting key 0a886c2778fd8e375eea4318110d9749
Sep 25 09:24:55 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328295, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:55.764110 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:55.737542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:60153 (local address 127.3.82.62:35127)
0925 09:24:55.738212 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:55.738219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:55.739570 (+ 1351us) server_negotiation.cc:407] Connection header received
0925 09:24:55.742900 (+ 3330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:55.742905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:55.742985 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:55.743093 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:55.745692 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:55.746657 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:55.749680 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:55.750586 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:55.750760 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:55.755475 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:55.755486 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:55.755495 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:55.755551 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:55.760022 (+ 4471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:55.761122 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:55.761132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:55.761138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:55.761307 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:55.761884 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:55.761892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:55.761899 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:55.762326 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:55.762499 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:55.763343 (+ 844us) server_negotiation.cc:299] Negotiation successful
0925 09:24:55.763658 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":332,"threads_started":1}
I20260925 09:24:55.765035 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:55.738343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35127 (local address 127.3.82.1:60153)
0925 09:24:55.739344 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:55.739411 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:55.742537 (+ 3126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:55.743433 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:55.743449 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:55.744384 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:55.745376 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:55.745410 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:55.746896 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:55.746905 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:55.749445 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:55.749471 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:55.750924 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:55.750932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:55.751189 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:55.752059 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:55.752086 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:55.755135 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:55.760273 (+ 5138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:55.760289 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:55.760312 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:55.760962 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:55.761494 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:55.761498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:55.761500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:55.761724 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:55.762535 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:55.762542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:55.762923 (+ 381us) client_negotiation.cc:769] Sending connection context
0925 09:24:55.763190 (+ 267us) client_negotiation.cc:240] Negotiation successful
0925 09:24:55.763759 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":367,"threads_started":1}
I20260925 09:24:55.766870 21870 heartbeater.cc:347] Connected to a master server at 127.3.82.62:35127
I20260925 09:24:55.767257 21870 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
I20260925 09:24:55.768275 21870 heartbeater.cc:511] Master 127.3.82.62:35127 requested a full tablet report, sending...
I20260925 09:24:55.771153 21681 ts_manager.cc:194] Registered new tserver with Master: 8fe67345abe44f93ba7e24de87c4c256 (127.3.82.1:36175)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
I20260925 09:24:55.773445 21681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:60153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:24:55.806370 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35127
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:56.165113 21876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:56.165498 21876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:56.165679 21876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:56.174544 21876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:56.174763 21876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:56.174896 21876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:24:56.185400 21876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:56.186861 21876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:56.188268 21876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:56.204025 21881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:56.203958 21884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:56.203994 21882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:56.204910 21876 server_base.cc:1034] running on GCE node
I20260925 09:24:56.205574 21876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:56.206667 21876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:56.207924 21876 hybrid_clock.cc:648] HybridClock initialized: now 1790328296207878 us; error 37 us; skew 500 ppm
I20260925 09:24:56.208334 21876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:56 dist-test-slave-4wm5 krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328296, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:56.212023 21876 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:24:56.213973 21876 webserver.cc:460] Webserver started at http://127.3.82.2:44141/ using document root <none> and password file <none>
I20260925 09:24:56.214733 21876 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:56.214879 21876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:56.215198 21876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:56.218374 21876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "e332ea50f0274589ac925128de13d6c6"
format_stamp: "Formatted at 2026-09-25 09:24:56 on dist-test-slave-4wm5"
server_key: "03c56328093bbff00e4b0ac2abb162ba"
server_key_iv: "082e6d117fd960449881bb3cb3c4462b"
server_key_version: "encryptionkey@0"
I20260925 09:24:56.219211 21876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "e332ea50f0274589ac925128de13d6c6"
format_stamp: "Formatted at 2026-09-25 09:24:56 on dist-test-slave-4wm5"
server_key: "03c56328093bbff00e4b0ac2abb162ba"
server_key_iv: "082e6d117fd960449881bb3cb3c4462b"
server_key_version: "encryptionkey@0"
I20260925 09:24:56.225036 21876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:24:56.229336 21891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:56.231091 21876 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:24:56.231316 21876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "e332ea50f0274589ac925128de13d6c6"
format_stamp: "Formatted at 2026-09-25 09:24:56 on dist-test-slave-4wm5"
server_key: "03c56328093bbff00e4b0ac2abb162ba"
server_key_iv: "082e6d117fd960449881bb3cb3c4462b"
server_key_version: "encryptionkey@0"
I20260925 09:24:56.231604 21876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:56.261767 21876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:56.267441 21876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:56.267834 21876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:56.269094 21876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:56.270710 21876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:56.270839 21876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:56.271000 21876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:56.271080 21876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:56.307952 21876 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:43199
I20260925 09:24:56.307953 22003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:43199 every 8 connection(s)
I20260925 09:24:56.309342 21876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:24:56.314929 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 21876
I20260925 09:24:56.315137 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:24:56.315721 3400 external_mini_cluster.cc:1428] Setting key 29ef4902231195da246120e8819b4890
Sep 25 09:24:56 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328296, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:56.335305 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.312399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:33109 (local address 127.3.82.62:35127)
0925 09:24:56.312900 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:56.312904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:56.313664 (+ 760us) server_negotiation.cc:407] Connection header received
0925 09:24:56.316513 (+ 2849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:56.316521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:56.316645 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:56.316739 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:56.319016 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.319767 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.322494 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.323041 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.323165 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.327912 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:56.327917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:56.327921 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:56.327960 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:56.331406 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:56.332403 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:56.332408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:56.332411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:56.332563 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:56.333118 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:56.333123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:56.333125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:56.333454 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:56.333654 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:56.334560 (+ 906us) server_negotiation.cc:299] Negotiation successful
0925 09:24:56.334861 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":256,"threads_started":1}
I20260925 09:24:56.336617 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.312711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35127 (local address 127.3.82.2:33109)
0925 09:24:56.313477 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:56.313516 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:56.316050 (+ 2534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:56.317142 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:56.317151 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:56.317883 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:56.318770 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.318800 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.319973 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.319982 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.322243 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.322262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.323313 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.323322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.323687 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.324585 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:56.324620 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:56.327599 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:56.331625 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:56.331637 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:56.331661 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:56.332220 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:56.332707 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:56.332714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:56.332719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:56.332966 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:56.333608 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:56.333620 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:56.334128 (+ 508us) client_negotiation.cc:769] Sending connection context
0925 09:24:56.334450 (+ 322us) client_negotiation.cc:240] Negotiation successful
0925 09:24:56.335110 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":272,"threads_started":1}
I20260925 09:24:56.338367 22004 heartbeater.cc:347] Connected to a master server at 127.3.82.62:35127
I20260925 09:24:56.338721 22004 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:56.339540 22004 heartbeater.cc:511] Master 127.3.82.62:35127 requested a full tablet report, sending...
I20260925 09:24:56.341212 21681 ts_manager.cc:194] Registered new tserver with Master: e332ea50f0274589ac925128de13d6c6 (127.3.82.2:43199)
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
I20260925 09:24:56.342283 21681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:33109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:24:56.378090 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35127
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:24:56.725636 22010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:56.725991 22010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:56.726097 22010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:56.733665 22010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:56.733937 22010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:56.734161 22010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:24:56.744575 22010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:56.746012 22010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:56.747385 22010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:56.761746 22015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:56.761770 22018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:56.761806 22016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:56.762419 22010 server_base.cc:1034] running on GCE node
I20260925 09:24:56.763352 22010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:56.764487 22010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:56.765715 22010 hybrid_clock.cc:648] HybridClock initialized: now 1790328296765649 us; error 54 us; skew 500 ppm
I20260925 09:24:56.766220 22010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:24:56 dist-test-slave-4wm5 krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328296, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:24:56.770117 22010 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:24:56.771916 22010 webserver.cc:460] Webserver started at http://127.3.82.3:33885/ using document root <none> and password file <none>
I20260925 09:24:56.772791 22010 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:56.772929 22010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:56.773343 22010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:56.776808 22010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "82b3e19e5b9d41e5bf0648a695450263"
format_stamp: "Formatted at 2026-09-25 09:24:56 on dist-test-slave-4wm5"
server_key: "8128c5e5047bd2ea6a2883126a46322b"
server_key_iv: "6c87cfa135193e9b5b22d4886b12fae2"
server_key_version: "encryptionkey@0"
I20260925 09:24:56.777503 21870 heartbeater.cc:503] Master 127.3.82.62:35127 was elected leader, sending a full tablet report...
I20260925 09:24:56.777706 22010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "82b3e19e5b9d41e5bf0648a695450263"
format_stamp: "Formatted at 2026-09-25 09:24:56 on dist-test-slave-4wm5"
server_key: "8128c5e5047bd2ea6a2883126a46322b"
server_key_iv: "6c87cfa135193e9b5b22d4886b12fae2"
server_key_version: "encryptionkey@0"
I20260925 09:24:56.783563 22010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 09:24:56.788097 22025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:56.789783 22010 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 09:24:56.790051 22010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "82b3e19e5b9d41e5bf0648a695450263"
format_stamp: "Formatted at 2026-09-25 09:24:56 on dist-test-slave-4wm5"
server_key: "8128c5e5047bd2ea6a2883126a46322b"
server_key_iv: "6c87cfa135193e9b5b22d4886b12fae2"
server_key_version: "encryptionkey@0"
I20260925 09:24:56.790334 22010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:56.807127 22010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:56.812007 22010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:56.812325 22010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:56.813349 22010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:56.814800 22010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:56.814911 22010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:56.815063 22010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:56.815140 22010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:56.850545 22010 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:37987
I20260925 09:24:56.850606 22137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:37987 every 8 connection(s)
I20260925 09:24:56.851902 22010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:24:56.855780 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22010
I20260925 09:24:56.855985 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:24:56.856568 3400 external_mini_cluster.cc:1428] Setting key ab02efcf2e51f8c04002a938406c1801
Sep 25 09:24:56 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328296, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:24:56.874133 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.854860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:40607 (local address 127.3.82.62:35127)
0925 09:24:56.855463 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:56.855468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:56.856368 (+ 900us) server_negotiation.cc:407] Connection header received
0925 09:24:56.858648 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:56.858653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:56.858772 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:56.858895 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:56.860704 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.861476 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.863146 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.863581 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.863723 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.867654 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:56.867659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:56.867662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:56.867697 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:56.870736 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:56.871592 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:56.871597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:56.871599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:56.871737 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:56.872211 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:56.872221 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:56.872223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:56.872511 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:56.872679 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:56.873426 (+ 747us) server_negotiation.cc:299] Negotiation successful
0925 09:24:56.873693 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":305,"threads_started":1}
I20260925 09:24:56.875209 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.855205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35127 (local address 127.3.82.3:40607)
0925 09:24:56.856162 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:56.856206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:56.858356 (+ 2150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:56.859159 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:56.859169 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:56.859876 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:56.860488 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.860506 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.861655 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.861659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.862995 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.863007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.863842 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.863850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.864092 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.864875 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:24:56.864904 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:56.867377 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:56.870912 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:56.870930 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:56.870947 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:56.871427 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:56.871867 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:24:56.871873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:24:56.871876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:24:56.872084 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:24:56.872655 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:56.872663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:56.873043 (+ 380us) client_negotiation.cc:769] Sending connection context
0925 09:24:56.873334 (+ 291us) client_negotiation.cc:240] Negotiation successful
0925 09:24:56.873957 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":367,"threads_started":1}
I20260925 09:24:56.876937 22138 heartbeater.cc:347] Connected to a master server at 127.3.82.62:35127
I20260925 09:24:56.877214 22138 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:56.878007 22138 heartbeater.cc:511] Master 127.3.82.62:35127 requested a full tablet report, sending...
I20260925 09:24:56.879485 21681 ts_manager.cc:194] Registered new tserver with Master: 82b3e19e5b9d41e5bf0648a695450263 (127.3.82.3:37987)
I20260925 09:24:56.880263 21681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:40607
I20260925 09:24:56.884925 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:24:56.904255 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.889415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.3.82.62:35127)
0925 09:24:56.889666 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:56.889676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:56.890078 (+ 402us) server_negotiation.cc:407] Connection header received
0925 09:24:56.890223 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:56.890227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:56.890305 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:56.890413 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:56.891887 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.892642 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.894083 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.894488 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.894590 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.897681 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:56.897686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:56.897690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:56.897764 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:56.900462 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:56.901660 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:56.901664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:56.901666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:56.901876 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:56.902455 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:56.902459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:56.902462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:56.902939 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:56.903167 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:56.903618 (+ 451us) server_negotiation.cc:299] Negotiation successful
0925 09:24:56.903784 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:24:56.908949 21681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:24:56.916314 21681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:24:56.946038 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.930587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.3.82.1:36175)
0925 09:24:56.931975 (+ 1388us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:56.931990 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:56.933317 (+ 1327us) server_negotiation.cc:407] Connection header received
0925 09:24:56.933714 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:56.933723 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:56.934084 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:56.934289 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:56.935698 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.937903 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.941703 (+ 3800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.943933 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.944078 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.944435 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:56.945405 (+ 970us) server_negotiation.cc:299] Negotiation successful
0925 09:24:56.945622 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":952,"threads_started":1}
I20260925 09:24:56.946816 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.930306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:36175 (local address 127.0.0.1:56780)
0925 09:24:56.933118 (+ 2812us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:56.933166 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:56.933454 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:56.934760 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:56.934771 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:56.934813 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:56.935444 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.935463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.938134 (+ 2671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.938142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.941424 (+ 3282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.941450 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.944708 (+ 3258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.944718 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.945054 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.945130 (+ 76us) client_negotiation.cc:769] Sending connection context
0925 09:24:56.946163 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0925 09:24:56.946400 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2524,"thread_start_us":2371,"threads_started":1}
I20260925 09:24:56.949875 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.930768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:37987 (local address 127.0.0.1:59628)
0925 09:24:56.933189 (+ 2421us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:56.933212 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:56.935039 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:56.936470 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:56.936477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:56.936514 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:56.937149 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.937167 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.941786 (+ 4619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.941795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.946610 (+ 4815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.946627 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.949030 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.949040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.949240 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.949273 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:24:56.949377 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 09:24:56.949515 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2301,"thread_start_us":2161,"threads_started":1}
I20260925 09:24:56.950577 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.931231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59628 (local address 127.3.82.3:37987)
0925 09:24:56.932742 (+ 1511us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:56.932754 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:56.933482 (+ 728us) server_negotiation.cc:407] Connection header received
0925 09:24:56.935408 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:56.935422 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:56.935775 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:56.936087 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:56.937459 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.941533 (+ 4074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.946824 (+ 5291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.948493 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.948627 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.948869 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:56.949577 (+ 708us) server_negotiation.cc:299] Negotiation successful
0925 09:24:56.949793 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296,"thread_start_us":1082,"threads_started":1}
I20260925 09:24:56.951284 21805 tablet_service.cc:1467] Processing CreateTablet for tablet 40d801a0a89f485983659ec9cd213325 (DEFAULT_TABLE table=test-table [id=517405ed1b7c43aeb5d939c169dbbe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:56.952908 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d801a0a89f485983659ec9cd213325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:56.954075 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.934017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.3.82.2:43199)
0925 09:24:56.946473 (+ 12456us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:56.946480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:56.946506 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:24:56.946589 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:56.946595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:56.946929 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:56.947195 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:56.948006 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.949560 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.951995 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.952956 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.953122 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.953319 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:56.953576 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 09:24:56.953743 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12002,"thread_start_us":11862,"threads_started":1}
I20260925 09:24:56.954382 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:56.933674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43199 (local address 127.0.0.1:50926)
0925 09:24:56.945296 (+ 11622us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:56.945322 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:56.945456 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:56.947452 (+ 1996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:56.947457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:56.947506 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:56.947816 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.947829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.949741 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.949747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.951821 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:56.951834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:56.953210 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:56.953216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:56.953383 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:56.953416 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:24:56.953531 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 09:24:56.953662 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11438,"thread_start_us":11858,"threads_started":1}
I20260925 09:24:56.954412 22073 tablet_service.cc:1467] Processing CreateTablet for tablet 40d801a0a89f485983659ec9cd213325 (DEFAULT_TABLE table=test-table [id=517405ed1b7c43aeb5d939c169dbbe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:56.955996 22073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d801a0a89f485983659ec9cd213325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:56.957913 21939 tablet_service.cc:1467] Processing CreateTablet for tablet 40d801a0a89f485983659ec9cd213325 (DEFAULT_TABLE table=test-table [id=517405ed1b7c43aeb5d939c169dbbe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:24:56.959110 21939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d801a0a89f485983659ec9cd213325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:56.969988 22157 tablet_bootstrap.cc:492] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Bootstrap starting.
I20260925 09:24:56.972719 22159 tablet_bootstrap.cc:492] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Bootstrap starting.
I20260925 09:24:56.973626 22158 tablet_bootstrap.cc:492] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Bootstrap starting.
I20260925 09:24:56.976145 22157 tablet_bootstrap.cc:654] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:56.976640 22159 tablet_bootstrap.cc:654] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:56.977157 22158 tablet_bootstrap.cc:654] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:56.977451 22157 log.cc:824] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:56.977973 22159 log.cc:824] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:56.978444 22158 log.cc:824] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:56.981413 22157 tablet_bootstrap.cc:492] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: No bootstrap required, opened a new log
I20260925 09:24:56.981586 22159 tablet_bootstrap.cc:492] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: No bootstrap required, opened a new log
I20260925 09:24:56.981662 22158 tablet_bootstrap.cc:492] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: No bootstrap required, opened a new log
I20260925 09:24:56.981813 22157 ts_tablet_manager.cc:1397] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260925 09:24:56.981907 22158 ts_tablet_manager.cc:1397] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260925 09:24:56.981930 22159 ts_tablet_manager.cc:1397] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260925 09:24:56.991166 22157 raft_consensus.cc:348] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:56.991708 22157 raft_consensus.cc:374] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:56.991819 22157 raft_consensus.cc:729] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe67345abe44f93ba7e24de87c4c256, State: Initialized, Role: FOLLOWER
I20260925 09:24:56.992588 22157 consensus_queue.cc:260] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:56.993319 22158 raft_consensus.cc:348] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:56.993907 22158 raft_consensus.cc:374] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:56.994023 22158 raft_consensus.cc:729] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b3e19e5b9d41e5bf0648a695450263, State: Initialized, Role: FOLLOWER
I20260925 09:24:56.994789 22157 ts_tablet_manager.cc:1428] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260925 09:24:56.994829 22158 consensus_queue.cc:260] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:56.993314 22159 raft_consensus.cc:348] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:56.995499 22159 raft_consensus.cc:374] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:56.995630 22159 raft_consensus.cc:729] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e332ea50f0274589ac925128de13d6c6, State: Initialized, Role: FOLLOWER
I20260925 09:24:56.996249 22138 heartbeater.cc:503] Master 127.3.82.62:35127 was elected leader, sending a full tablet report...
I20260925 09:24:56.996382 22159 consensus_queue.cc:260] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:56.996879 22158 ts_tablet_manager.cc:1428] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260925 09:24:56.997987 22004 heartbeater.cc:503] Master 127.3.82.62:35127 was elected leader, sending a full tablet report...
I20260925 09:24:57.000036 22159 ts_tablet_manager.cc:1428] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260925 09:24:57.061628 22005 tablet.cc:2369] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:24:57.093170 22164 raft_consensus.cc:482] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:24:57.093464 22164 raft_consensus.cc:504] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:57.095549 22164 leader_election.cc:288] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe67345abe44f93ba7e24de87c4c256 (127.3.82.1:36175), e332ea50f0274589ac925128de13d6c6 (127.3.82.2:43199)
I20260925 09:24:57.103220 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.095986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:40617 (local address 127.3.82.1:36175)
0925 09:24:57.096200 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:57.096208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:57.096301 (+ 93us) server_negotiation.cc:407] Connection header received
0925 09:24:57.096637 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:57.096645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:57.096766 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:57.096902 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:57.097418 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.098603 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.100830 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.102193 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.102358 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.102487 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:57.102772 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 09:24:57.102929 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 09:24:57.103255 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.095929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:36175 (local address 127.3.82.3:40617)
0925 09:24:57.096192 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:57.096213 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:57.096444 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:57.096943 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:57.096954 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:57.096990 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:57.097204 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:57.097217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.098769 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.098773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:57.100641 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:57.100656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.102442 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.102451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:57.102594 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.102620 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 09:24:57.102701 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 09:24:57.102880 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
W20260925 09:24:57.103787 22139 tablet.cc:2369] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:24:57.103883 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.096546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:59443 (local address 127.3.82.2:43199)
0925 09:24:57.096784 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:57.096792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:57.097364 (+ 572us) server_negotiation.cc:407] Connection header received
0925 09:24:57.097558 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:57.097564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:57.097651 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:57.097770 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:24:57.098430 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.099277 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.101494 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.102827 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.102972 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.103074 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:57.103473 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 09:24:57.103622 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:24:57.104413 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d801a0a89f485983659ec9cd213325" candidate_uuid: "82b3e19e5b9d41e5bf0648a695450263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe67345abe44f93ba7e24de87c4c256" is_pre_election: true
I20260925 09:24:57.104656 22166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.096385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:43199 (local address 127.3.82.3:59443)
0925 09:24:57.097205 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:57.097227 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:57.097418 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:57.097912 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:57.097915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:57.097937 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:24:57.098254 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:57.098262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.099454 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.099460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:57.101326 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:57.101338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.103059 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.103065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:57.103234 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.103276 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 09:24:57.104118 (+ 842us) client_negotiation.cc:240] Negotiation successful
0925 09:24:57.104277 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":505,"threads_started":1}
I20260925 09:24:57.105017 21825 raft_consensus.cc:2454] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82b3e19e5b9d41e5bf0648a695450263 in term 0.
I20260925 09:24:57.105772 22027 leader_election.cc:302] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82b3e19e5b9d41e5bf0648a695450263, 8fe67345abe44f93ba7e24de87c4c256; no voters:
I20260925 09:24:57.105938 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d801a0a89f485983659ec9cd213325" candidate_uuid: "82b3e19e5b9d41e5bf0648a695450263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e332ea50f0274589ac925128de13d6c6" is_pre_election: true
I20260925 09:24:57.106297 22164 raft_consensus.cc:2781] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:24:57.106455 22164 raft_consensus.cc:482] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:24:57.106575 22164 raft_consensus.cc:3037] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:57.106611 21959 raft_consensus.cc:2454] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82b3e19e5b9d41e5bf0648a695450263 in term 0.
I20260925 09:24:57.109186 22164 raft_consensus.cc:504] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:57.109738 22164 leader_election.cc:288] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe67345abe44f93ba7e24de87c4c256 (127.3.82.1:36175), e332ea50f0274589ac925128de13d6c6 (127.3.82.2:43199)
I20260925 09:24:57.110533 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d801a0a89f485983659ec9cd213325" candidate_uuid: "82b3e19e5b9d41e5bf0648a695450263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e332ea50f0274589ac925128de13d6c6"
I20260925 09:24:57.110538 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d801a0a89f485983659ec9cd213325" candidate_uuid: "82b3e19e5b9d41e5bf0648a695450263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe67345abe44f93ba7e24de87c4c256"
I20260925 09:24:57.110778 21959 raft_consensus.cc:3037] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:57.110782 21825 raft_consensus.cc:3037] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:57.114176 21959 raft_consensus.cc:2454] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82b3e19e5b9d41e5bf0648a695450263 in term 1.
I20260925 09:24:57.114184 21825 raft_consensus.cc:2454] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82b3e19e5b9d41e5bf0648a695450263 in term 1.
I20260925 09:24:57.114679 22027 leader_election.cc:302] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82b3e19e5b9d41e5bf0648a695450263, 8fe67345abe44f93ba7e24de87c4c256; no voters:
I20260925 09:24:57.115003 22164 raft_consensus.cc:2781] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:57.115506 22164 raft_consensus.cc:686] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 1 LEADER]: Becoming Leader. State: Replica: 82b3e19e5b9d41e5bf0648a695450263, State: Running, Role: LEADER
I20260925 09:24:57.116080 22164 consensus_queue.cc:237] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } }
I20260925 09:24:57.121006 21680 catalog_manager.cc:5495] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82b3e19e5b9d41e5bf0648a695450263 (127.3.82.3). New cstate: current_term: 1 leader_uuid: "82b3e19e5b9d41e5bf0648a695450263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82b3e19e5b9d41e5bf0648a695450263" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 37987 } health_report { overall_health: HEALTHY } } }
I20260925 09:24:57.144423 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.137460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.3.82.3:37987)
0925 09:24:57.137674 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:57.137679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:57.137698 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:24:57.137803 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:57.137809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:57.137972 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:57.138115 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:24:57.138512 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.139327 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.140955 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.141488 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.141642 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.142655 (+ 1013us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:24:57.143481 (+ 826us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:24:57.143711 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:57.143973 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 09:24:57.144037 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:24:57.156108 21959 raft_consensus.cc:1261] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 1 FOLLOWER]: Refusing update from remote peer 82b3e19e5b9d41e5bf0648a695450263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:57.156127 21825 raft_consensus.cc:1261] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 1 FOLLOWER]: Refusing update from remote peer 82b3e19e5b9d41e5bf0648a695450263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:24:57.157177 22164 consensus_queue.cc:1035] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe67345abe44f93ba7e24de87c4c256" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 36175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:57.157528 22167 consensus_queue.cc:1035] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e332ea50f0274589ac925128de13d6c6" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 43199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:24:57.190111 21681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:57.190467 21681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:24:57.194514 21681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b92e80846df46baa4d23f993383a9b0: Sending DeleteTablet for 3 replicas of tablet 40d801a0a89f485983659ec9cd213325
I20260925 09:24:57.196070 22073 tablet_service.cc:1514] Processing DeleteTablet for tablet 40d801a0a89f485983659ec9cd213325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:57 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:59628
I20260925 09:24:57.196103 21939 tablet_service.cc:1514] Processing DeleteTablet for tablet 40d801a0a89f485983659ec9cd213325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:57 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:50926
I20260925 09:24:57.196607 21805 tablet_service.cc:1514] Processing DeleteTablet for tablet 40d801a0a89f485983659ec9cd213325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:24:57 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:56780
I20260925 09:24:57.201583 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21742
I20260925 09:24:57.214020 22181 tablet_replica.cc:330] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: stopping tablet replica
I20260925 09:24:57.214687 22183 tablet_replica.cc:330] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: stopping tablet replica
I20260925 09:24:57.215420 22183 raft_consensus.cc:2229] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:57.215967 22181 raft_consensus.cc:2229] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:24:57.216624 22183 raft_consensus.cc:2258] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:57.217365 22181 raft_consensus.cc:2258] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:57.220136 22183 ts_tablet_manager.cc:1905] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:57.221660 22181 ts_tablet_manager.cc:1905] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:24:57.222342 22182 tablet_replica.cc:330] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: stopping tablet replica
I20260925 09:24:57.222795 22182 raft_consensus.cc:2229] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:24:57.223326 22182 raft_consensus.cc:2258] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:24:57.225488 22182 ts_tablet_manager.cc:1905] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 09:24:57 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:24:57.230115 22181 ts_tablet_manager.cc:1918] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:57.230321 22181 log.cc:1196] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/40d801a0a89f485983659ec9cd213325
I20260925 09:24:57.230928 22181 ts_tablet_manager.cc:1939] T 40d801a0a89f485983659ec9cd213325 P 82b3e19e5b9d41e5bf0648a695450263: Deleting consensus metadata
I20260925 09:24:57.233121 21667 catalog_manager.cc:4841] TS 82b3e19e5b9d41e5bf0648a695450263 (127.3.82.3:37987): tablet 40d801a0a89f485983659ec9cd213325 (table test-table [id=517405ed1b7c43aeb5d939c169dbbe7a]) successfully deleted
I20260925 09:24:57.233791 22182 ts_tablet_manager.cc:1918] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:57.234048 22182 log.cc:1196] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/40d801a0a89f485983659ec9cd213325
I20260925 09:24:57.234607 22182 ts_tablet_manager.cc:1939] T 40d801a0a89f485983659ec9cd213325 P e332ea50f0274589ac925128de13d6c6: Deleting consensus metadata
I20260925 09:24:57.236688 21666 catalog_manager.cc:4841] TS e332ea50f0274589ac925128de13d6c6 (127.3.82.2:43199): tablet 40d801a0a89f485983659ec9cd213325 (table test-table [id=517405ed1b7c43aeb5d939c169dbbe7a]) successfully deleted
I20260925 09:24:57.239658 22183 ts_tablet_manager.cc:1918] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:24:57.239863 22183 log.cc:1196] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/40d801a0a89f485983659ec9cd213325
I20260925 09:24:57.240360 22183 ts_tablet_manager.cc:1939] T 40d801a0a89f485983659ec9cd213325 P 8fe67345abe44f93ba7e24de87c4c256: Deleting consensus metadata
I20260925 09:24:57.242445 21666 catalog_manager.cc:4841] TS 8fe67345abe44f93ba7e24de87c4c256 (127.3.82.1:36175): tablet 40d801a0a89f485983659ec9cd213325 (table test-table [id=517405ed1b7c43aeb5d939c169dbbe7a]) successfully deleted
I20260925 09:24:57.243372 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.203016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.3.82.1:36175)
0925 09:24:57.203268 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:57.203277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:57.211755 (+ 8478us) server_negotiation.cc:407] Connection header received
0925 09:24:57.211970 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:57.211979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:57.212152 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:57.212305 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:57.215959 (+ 3654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.217581 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.221243 (+ 3662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.222132 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.222236 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.234491 (+ 12255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:57.234507 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:57.234528 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:57.234590 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:57.239591 (+ 5001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:57.240732 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:57.240742 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:57.240745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:57.240945 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:57.241469 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:57.241476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:57.241480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:57.242017 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:57.242325 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:57.242783 (+ 458us) server_negotiation.cc:299] Negotiation successful
0925 09:24:57.243007 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 09:24:57.244076 21865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:57.350684 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21742
I20260925 09:24:57.376703 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21876
Sep 25 09:24:57 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:24:57.396524 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.377575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.3.82.2:43199)
0925 09:24:57.377779 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:57.377787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:57.377813 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:24:57.378002 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:57.378009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:57.378112 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:57.378233 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:57.379771 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.380985 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.382595 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.383233 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.383381 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.388553 (+ 5172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:57.388576 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:57.388597 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:57.388657 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:57.392994 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:57.394012 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:57.394025 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:57.394028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:57.394201 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:57.394754 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:57.394761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:57.394764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:57.395270 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:57.395575 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:57.395887 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 09:24:57.396120 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 09:24:57.397441 21999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:57.492926 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21876
I20260925 09:24:57.518093 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22010
Sep 25 09:24:57 dist-test-slave-4wm5 krb5kdc[21636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:24:57.538475 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:57.519050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.3.82.3:37987)
0925 09:24:57.519223 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:57.519228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:57.519335 (+ 107us) server_negotiation.cc:407] Connection header received
0925 09:24:57.519554 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:57.519559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:57.519640 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:57.519768 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:57.521566 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.522499 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.524524 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:57.525013 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:57.525124 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:57.531491 (+ 6367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:57.531509 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:57.531526 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:24:57.531568 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:57.535114 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:57.536022 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:57.536030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:57.536033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:57.536208 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:24:57.536631 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:24:57.536637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:24:57.536640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:24:57.537035 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:57.537418 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:57.537778 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 09:24:57.538076 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 09:24:57.539183 22133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:57.671663 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22010
I20260925 09:24:57.692050 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 21649
I20260925 09:24:57.693377 21711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:24:57.808234 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 21649
2026-09-25T09:24:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4660 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T09:24:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:24:57Z Disabled control of system clock
I20260925 09:24:57.890080 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37063
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:43003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:24:58.252684 22192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:58.253017 22192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:58.253101 22192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:58.260747 22192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:24:58.260972 22192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:58.261014 22192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:58.261065 22192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:24:58.261112 22192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:24:58.270694 22192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37063
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:58.272020 22192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:58.273861 22192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:58.286185 22197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:58.286262 22200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:58.286286 22198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:58.287552 22192 server_base.cc:1034] running on GCE node
I20260925 09:24:58.288555 22192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:58.290064 22192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:58.291330 22192 hybrid_clock.cc:648] HybridClock initialized: now 1790328298291283 us; error 57 us; skew 500 ppm
I20260925 09:24:58.291838 22192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:58.294318 22192 webserver.cc:460] Webserver started at http://127.3.82.62:44723/ using document root <none> and password file <none>
I20260925 09:24:58.295133 22192 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:58.295274 22192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:58.295611 22192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:58.299044 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "67465338bd9d44e9804a89891b25a544"
format_stamp: "Formatted at 2026-09-25 09:24:58 on dist-test-slave-4wm5"
server_key: "6024b58e6fdba1b7b360bd84d6328685"
server_key_iv: "e1cb73f3aca2351dadc969d37662c3eb"
server_key_version: "encryptionkey@0"
I20260925 09:24:58.299957 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "67465338bd9d44e9804a89891b25a544"
format_stamp: "Formatted at 2026-09-25 09:24:58 on dist-test-slave-4wm5"
server_key: "6024b58e6fdba1b7b360bd84d6328685"
server_key_iv: "e1cb73f3aca2351dadc969d37662c3eb"
server_key_version: "encryptionkey@0"
I20260925 09:24:58.305603 22192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:24:58.310122 22206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:58.311888 22192 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:24:58.312155 22192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "67465338bd9d44e9804a89891b25a544"
format_stamp: "Formatted at 2026-09-25 09:24:58 on dist-test-slave-4wm5"
server_key: "6024b58e6fdba1b7b360bd84d6328685"
server_key_iv: "e1cb73f3aca2351dadc969d37662c3eb"
server_key_version: "encryptionkey@0"
I20260925 09:24:58.312431 22192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:58.331353 22192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:58.332347 22192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:58.332665 22192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:58.353286 22192 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37063
I20260925 09:24:58.353343 22257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37063 every 8 connection(s)
I20260925 09:24:58.354748 22192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:24:58.357565 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22192
I20260925 09:24:58.357774 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:24:58.358275 3400 external_mini_cluster.cc:1428] Setting key 4a0e9fa445f18b9d994a97aefc18acaf
I20260925 09:24:58.359181 22258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:24:58.374812 22258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544: Bootstrap starting.
I20260925 09:24:58.375452 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:58.360283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.3.82.62:37063)
0925 09:24:58.361530 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:58.361542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:58.361588 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:24:58.364498 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:58.364522 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:58.365087 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:58.365461 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:58.366168 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:58.368196 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:58.370282 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:58.371056 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:58.371212 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:58.372644 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:58.372653 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:58.372673 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:58.373393 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:58.373481 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:58.373505 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:58.373614 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:58.374001 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 09:24:58.374385 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":423,"threads_started":1}
I20260925 09:24:58.378732 22258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544: Neither blocks nor log segments found. Creating new log.
I20260925 09:24:58.379787 22258 log.cc:824] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544: Log is configured to *not* fsync() on all Append() calls
I20260925 09:24:58.382791 22258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544: No bootstrap required, opened a new log
I20260925 09:24:58.391258 22258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67465338bd9d44e9804a89891b25a544" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37063 } }
I20260925 09:24:58.391671 22258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:24:58.391803 22258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67465338bd9d44e9804a89891b25a544, State: Initialized, Role: FOLLOWER
I20260925 09:24:58.392482 22258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67465338bd9d44e9804a89891b25a544" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37063 } }
I20260925 09:24:58.392694 22258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:24:58.392850 22258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:24:58.393024 22258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:24:58.395426 22258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67465338bd9d44e9804a89891b25a544" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37063 } }
I20260925 09:24:58.395972 22258 leader_election.cc:302] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67465338bd9d44e9804a89891b25a544; no voters:
I20260925 09:24:58.396615 22258 leader_election.cc:288] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:24:58.396788 22263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:24:58.397619 22263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [term 1 LEADER]: Becoming Leader. State: Replica: 67465338bd9d44e9804a89891b25a544, State: Running, Role: LEADER
I20260925 09:24:58.398208 22263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67465338bd9d44e9804a89891b25a544" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37063 } }
I20260925 09:24:58.399020 22258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:24:58.401543 22265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67465338bd9d44e9804a89891b25a544. Latest consensus state: current_term: 1 leader_uuid: "67465338bd9d44e9804a89891b25a544" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67465338bd9d44e9804a89891b25a544" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37063 } } }
I20260925 09:24:58.401469 22264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67465338bd9d44e9804a89891b25a544" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67465338bd9d44e9804a89891b25a544" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37063 } } }
I20260925 09:24:58.401984 22265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:58.401986 22264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544 [sys.catalog]: This master's current role is: LEADER
I20260925 09:24:58.403007 22271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:24:58.408772 22271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:24:58.418292 22271 catalog_manager.cc:1347] Generated new cluster ID: e345bf00110c44f9a069abf24fc3aabf
I20260925 09:24:58.418437 22271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:24:58.454033 22271 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:24:58.456018 22271 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:24:58.464301 22271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544: Generated new TSK 0
I20260925 09:24:58.465445 22271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:24:58.472379 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37063
--builtin_ntp_servers=127.3.82.20:43003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:24:58.830907 22282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:58.831245 22282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:58.831403 22282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:58.839468 22282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:58.839594 22282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:58.839767 22282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:24:58.849596 22282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:58.851089 22282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:58.852986 22282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:58.867177 22287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:58.867206 22288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:58.867476 22290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:58.868695 22282 server_base.cc:1034] running on GCE node
I20260925 09:24:58.869529 22282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:58.870806 22282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:58.872048 22282 hybrid_clock.cc:648] HybridClock initialized: now 1790328298871964 us; error 77 us; skew 500 ppm
I20260925 09:24:58.872442 22282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:58.875679 22282 webserver.cc:460] Webserver started at http://127.3.82.1:36653/ using document root <none> and password file <none>
I20260925 09:24:58.876524 22282 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:58.876618 22282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:58.876936 22282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:58.880112 22282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "fa2916b606374a6a9e5361cd4eeb0fde"
format_stamp: "Formatted at 2026-09-25 09:24:58 on dist-test-slave-4wm5"
server_key: "35482974bf6614f5fa86385871cb5b59"
server_key_iv: "04c953ebd58f01d16d3cbd705fb6c1bc"
server_key_version: "encryptionkey@0"
I20260925 09:24:58.880966 22282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "fa2916b606374a6a9e5361cd4eeb0fde"
format_stamp: "Formatted at 2026-09-25 09:24:58 on dist-test-slave-4wm5"
server_key: "35482974bf6614f5fa86385871cb5b59"
server_key_iv: "04c953ebd58f01d16d3cbd705fb6c1bc"
server_key_version: "encryptionkey@0"
I20260925 09:24:58.887072 22282 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 09:24:58.891376 22296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:58.893097 22282 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:24:58.893378 22282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "fa2916b606374a6a9e5361cd4eeb0fde"
format_stamp: "Formatted at 2026-09-25 09:24:58 on dist-test-slave-4wm5"
server_key: "35482974bf6614f5fa86385871cb5b59"
server_key_iv: "04c953ebd58f01d16d3cbd705fb6c1bc"
server_key_version: "encryptionkey@0"
I20260925 09:24:58.893709 22282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:58.912295 22282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:58.913298 22282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:58.913635 22282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:58.914855 22282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:58.916729 22282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:58.916889 22282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:58.917053 22282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:58.917158 22282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:58.956292 22282 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:40817
I20260925 09:24:58.956393 22408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:40817 every 8 connection(s)
I20260925 09:24:58.957813 22282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:24:58.962770 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22282
I20260925 09:24:58.962967 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:24:58.963536 3400 external_mini_cluster.cc:1428] Setting key 1f62035e954c3edfd0ac12725be17173
I20260925 09:24:58.966836 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37063
--builtin_ntp_servers=127.3.82.20:43003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:24:58.972790 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:58.960904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:43869 (local address 127.3.82.62:37063)
0925 09:24:58.961427 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:58.961434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:58.964200 (+ 2766us) server_negotiation.cc:407] Connection header received
0925 09:24:58.965476 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:58.965482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:58.965592 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:58.965710 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:58.967240 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:58.967973 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:58.969683 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:58.970168 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:58.970323 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:58.971866 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:58.971871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:58.971875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:58.971896 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:58.971939 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:58.971951 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:58.972051 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:58.972293 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 09:24:58.972404 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":239,"threads_started":1}
I20260925 09:24:58.974084 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:58.961419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37063 (local address 127.3.82.1:43869)
0925 09:24:58.962783 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:58.962841 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:58.965146 (+ 2305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:58.966016 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:58.966026 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:58.966132 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:58.966804 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:58.966824 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:58.968164 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:58.968169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:58.969508 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:58.969522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:58.970424 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:58.970431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:58.970682 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:58.971485 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:24:58.971487 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:58.971531 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:24:58.971622 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:58.972051 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:58.972065 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:58.972076 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 09:24:58.972286 (+ 210us) client_negotiation.cc:240] Negotiation successful
0925 09:24:58.972750 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":464,"threads_started":1}
I20260925 09:24:58.976229 22409 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37063
I20260925 09:24:58.976641 22409 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:58.977684 22409 heartbeater.cc:511] Master 127.3.82.62:37063 requested a full tablet report, sending...
I20260925 09:24:58.980235 22223 ts_manager.cc:194] Registered new tserver with Master: fa2916b606374a6a9e5361cd4eeb0fde (127.3.82.1:40817)
I20260925 09:24:58.981966 22223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.1:43869
W20260925 09:24:59.322314 22413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:59.322679 22413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:59.322842 22413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:59.330440 22413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:59.330614 22413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:59.330806 22413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:24:59.340461 22413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:59.341907 22413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:59.343864 22413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:59.358234 22418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:59.358278 22419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:59.358791 22421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:59.359699 22413 server_base.cc:1034] running on GCE node
I20260925 09:24:59.360471 22413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:59.361702 22413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:59.362964 22413 hybrid_clock.cc:648] HybridClock initialized: now 1790328299362910 us; error 69 us; skew 500 ppm
I20260925 09:24:59.363348 22413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:59.366534 22413 webserver.cc:460] Webserver started at http://127.3.82.2:34017/ using document root <none> and password file <none>
I20260925 09:24:59.367282 22413 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:59.367411 22413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:59.367744 22413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:59.370723 22413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "ea846612765641d98bc9d673c49560bd"
format_stamp: "Formatted at 2026-09-25 09:24:59 on dist-test-slave-4wm5"
server_key: "d031cc600b006d392fd4302eb34c3767"
server_key_iv: "729f7f8afd8623a7095d866a15a3e162"
server_key_version: "encryptionkey@0"
I20260925 09:24:59.371546 22413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "ea846612765641d98bc9d673c49560bd"
format_stamp: "Formatted at 2026-09-25 09:24:59 on dist-test-slave-4wm5"
server_key: "d031cc600b006d392fd4302eb34c3767"
server_key_iv: "729f7f8afd8623a7095d866a15a3e162"
server_key_version: "encryptionkey@0"
I20260925 09:24:59.377243 22413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:24:59.381762 22427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:59.383589 22413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:24:59.383847 22413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "ea846612765641d98bc9d673c49560bd"
format_stamp: "Formatted at 2026-09-25 09:24:59 on dist-test-slave-4wm5"
server_key: "d031cc600b006d392fd4302eb34c3767"
server_key_iv: "729f7f8afd8623a7095d866a15a3e162"
server_key_version: "encryptionkey@0"
I20260925 09:24:59.384112 22413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:59.405570 22413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:59.406549 22413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:59.406903 22413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:59.408016 22413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:59.409727 22413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:59.409901 22413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:59.410043 22413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:59.410137 22413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:59.447650 22413 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:40489
I20260925 09:24:59.447640 22539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:40489 every 8 connection(s)
I20260925 09:24:59.449039 22413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:24:59.458221 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22413
I20260925 09:24:59.458438 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:24:59.459007 3400 external_mini_cluster.cc:1428] Setting key fa1be64a212a471305fe1a0499661d4d
I20260925 09:24:59.462620 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37063
--builtin_ntp_servers=127.3.82.20:43003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:24:59.464522 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:59.452267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:40055 (local address 127.3.82.62:37063)
0925 09:24:59.452542 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:59.452547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:59.454238 (+ 1691us) server_negotiation.cc:407] Connection header received
0925 09:24:59.456144 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:59.456150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:59.456251 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:59.456368 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:59.457478 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.458279 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.460773 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.461261 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.461438 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:59.463538 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:59.463545 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:59.463549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:59.463572 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:59.463621 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:59.463636 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:59.463733 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:59.464066 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 09:24:59.464166 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260925 09:24:59.466159 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:59.452880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37063 (local address 127.3.82.2:40055)
0925 09:24:59.454074 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:59.454113 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:59.455843 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:59.456604 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:59.456612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:59.456706 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:59.457304 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:59.457321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.458492 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.458499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:59.460600 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:59.460617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.461632 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.461642 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:59.462072 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:59.463045 (+ 973us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:24:59.463051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:59.463116 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:24:59.463248 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:59.463782 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:59.463799 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:59.463811 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 09:24:59.464037 (+ 226us) client_negotiation.cc:240] Negotiation successful
0925 09:24:59.464578 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":397,"threads_started":1}
I20260925 09:24:59.467888 22540 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37063
I20260925 09:24:59.468257 22540 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:59.469308 22540 heartbeater.cc:511] Master 127.3.82.62:37063 requested a full tablet report, sending...
I20260925 09:24:59.471118 22223 ts_manager.cc:194] Registered new tserver with Master: ea846612765641d98bc9d673c49560bd (127.3.82.2:40489)
I20260925 09:24:59.472167 22223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.2:40055
W20260925 09:24:59.814706 22543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:24:59.815066 22543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:24:59.815173 22543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:24:59.823746 22543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:24:59.823920 22543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:24:59.824044 22543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:24:59.834688 22543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:37063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:24:59.836143 22543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:24:59.838260 22543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:24:59.854952 22548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:59.855005 22549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:24:59.854907 22551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:24:59.860167 22543 server_base.cc:1034] running on GCE node
I20260925 09:24:59.860988 22543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:24:59.862272 22543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:24:59.863507 22543 hybrid_clock.cc:648] HybridClock initialized: now 1790328299863462 us; error 42 us; skew 500 ppm
I20260925 09:24:59.863910 22543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:24:59.866581 22543 webserver.cc:460] Webserver started at http://127.3.82.3:36311/ using document root <none> and password file <none>
I20260925 09:24:59.867340 22543 fs_manager.cc:359] Metadata directory not provided
I20260925 09:24:59.867487 22543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:24:59.867877 22543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:24:59.871248 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "f70fe9e1d76842af9eaa4b239c3f39fd"
format_stamp: "Formatted at 2026-09-25 09:24:59 on dist-test-slave-4wm5"
server_key: "c35b63233409308af356c46fd1fde2a2"
server_key_iv: "5c72228e464785164496b62dcf077d8f"
server_key_version: "encryptionkey@0"
I20260925 09:24:59.872401 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "f70fe9e1d76842af9eaa4b239c3f39fd"
format_stamp: "Formatted at 2026-09-25 09:24:59 on dist-test-slave-4wm5"
server_key: "c35b63233409308af356c46fd1fde2a2"
server_key_iv: "5c72228e464785164496b62dcf077d8f"
server_key_version: "encryptionkey@0"
I20260925 09:24:59.878427 22543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260925 09:24:59.882594 22557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:59.884330 22543 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260925 09:24:59.884521 22543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "f70fe9e1d76842af9eaa4b239c3f39fd"
format_stamp: "Formatted at 2026-09-25 09:24:59 on dist-test-slave-4wm5"
server_key: "c35b63233409308af356c46fd1fde2a2"
server_key_iv: "5c72228e464785164496b62dcf077d8f"
server_key_version: "encryptionkey@0"
I20260925 09:24:59.884778 22543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:24:59.902367 22543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:24:59.903306 22543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:24:59.903648 22543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:24:59.904855 22543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:24:59.906699 22543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:24:59.906819 22543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:59.906971 22543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:24:59.907078 22543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:24:59.944520 22543 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:46309
I20260925 09:24:59.944643 22669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:46309 every 8 connection(s)
I20260925 09:24:59.945981 22543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:24:59.953919 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22543
I20260925 09:24:59.954128 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:24:59.954692 3400 external_mini_cluster.cc:1428] Setting key e97149091e231aa0d97cee45fbd7c888
I20260925 09:24:59.962630 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:59.949025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:53121 (local address 127.3.82.62:37063)
0925 09:24:59.949272 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:59.949278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:59.950971 (+ 1693us) server_negotiation.cc:407] Connection header received
0925 09:24:59.953255 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:59.953260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:59.953341 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:59.953441 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:59.955607 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.956667 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.959431 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.959902 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.960057 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:59.961660 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:59.961667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:59.961672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:59.961693 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:59.961733 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:59.961746 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:59.961874 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:59.962144 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 09:24:59.962259 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:24:59.963701 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:59.949434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:37063 (local address 127.3.82.3:53121)
0925 09:24:59.950654 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0925 09:24:59.950696 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 09:24:59.952882 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:24:59.954340 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:24:59.954354 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:24:59.954500 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:24:59.955397 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:59.955429 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.957651 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.957660 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:59.959236 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:24:59.959253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.960168 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.960176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:24:59.960428 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:59.961273 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:24:59.961275 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:24:59.961319 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:24:59.961417 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:24:59.961905 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:24:59.961920 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:24:59.961927 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:24:59.962115 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 09:24:59.962571 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":422,"threads_started":1}
I20260925 09:24:59.965363 22670 heartbeater.cc:347] Connected to a master server at 127.3.82.62:37063
I20260925 09:24:59.965731 22670 heartbeater.cc:464] Registering TS with master...
I20260925 09:24:59.966662 22670 heartbeater.cc:511] Master 127.3.82.62:37063 requested a full tablet report, sending...
I20260925 09:24:59.968608 22223 ts_manager.cc:194] Registered new tserver with Master: f70fe9e1d76842af9eaa4b239c3f39fd (127.3.82.3:46309)
I20260925 09:24:59.969650 22223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.3:53121
I20260925 09:24:59.971050 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:24:59.986392 22409 heartbeater.cc:503] Master 127.3.82.62:37063 was elected leader, sending a full tablet report...
I20260925 09:24:59.988162 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:24:59.975522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33408 (local address 127.3.82.62:37063)
0925 09:24:59.975781 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 09:24:59.975790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:24:59.976429 (+ 639us) server_negotiation.cc:407] Connection header received
0925 09:24:59.976663 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:24:59.976669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:24:59.976754 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:24:59.976874 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:24:59.977429 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.978341 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.979942 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:24:59.980415 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:24:59.980566 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:24:59.986941 (+ 6375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:24:59.986947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:24:59.986951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:24:59.986994 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:24:59.987042 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:24:59.987058 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:24:59.987261 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 09:24:59.987602 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 09:24:59.987693 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:24:59.993579 22223 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:25:00.003322 22223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:25:00.032779 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.018716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:46309 (local address 127.0.0.1:57326)
0925 09:25:00.020697 (+ 1981us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:00.020751 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:00.020924 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:00.024712 (+ 3788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:00.024719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:00.024745 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:00.025355 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.025372 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.027515 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.027521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.029787 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.029803 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.031641 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.031648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.031904 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.031970 (+ 66us) client_negotiation.cc:769] Sending connection context
0925 09:25:00.032090 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 09:25:00.032305 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1700,"mutex_wait_us":275,"spinlock_wait_cycles":896,"thread_start_us":1527,"threads_started":1}
I20260925 09:25:00.033464 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.019186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:40489 (local address 127.0.0.1:57970)
0925 09:25:00.023353 (+ 4167us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:00.023373 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:00.023503 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:00.024389 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:00.024397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:00.024434 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:00.025138 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.025173 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.027535 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.027541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.030625 (+ 3084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.030634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.032574 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.032581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.032781 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.032828 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 09:25:00.032922 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 09:25:00.033069 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2804,"mutex_wait_us":272,"thread_start_us":2660,"threads_started":1}
I20260925 09:25:00.033532 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.019918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.3.82.2:40489)
0925 09:25:00.021667 (+ 1749us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.021679 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.023654 (+ 1975us) server_negotiation.cc:407] Connection header received
0925 09:25:00.023786 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.023810 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.024126 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.024344 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:00.025423 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.027325 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.030903 (+ 3578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.032381 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.032565 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.032836 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.032979 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.033160 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1580,"thread_start_us":1470,"threads_started":1}
I20260925 09:25:00.034420 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.021926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40817 (local address 127.0.0.1:35460)
0925 09:25:00.023494 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:00.023522 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:00.023811 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:00.024643 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:00.024654 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:00.024685 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:00.025441 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.025485 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.027536 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.027541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.030616 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.030632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.033337 (+ 2705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.033343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.033524 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.033554 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 09:25:00.033939 (+ 385us) client_negotiation.cc:240] Negotiation successful
0925 09:25:00.034078 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"mutex_wait_us":209,"thread_start_us":1112,"threads_started":1}
I20260925 09:25:00.034534 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.018947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.3.82.3:46309)
0925 09:25:00.024156 (+ 5209us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.024166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.024199 (+ 33us) server_negotiation.cc:407] Connection header received
0925 09:25:00.024310 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.024318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.024512 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.024861 (+ 349us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:00.025572 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.027314 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.030084 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.031410 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.033402 (+ 1992us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.033685 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.033971 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.034148 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5041,"thread_start_us":4928,"threads_started":1}
I20260925 09:25:00.034530 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.021502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.3.82.1:40817)
0925 09:25:00.022830 (+ 1328us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.022842 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.023676 (+ 834us) server_negotiation.cc:407] Connection header received
0925 09:25:00.024034 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.024042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.024308 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.024510 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:00.025674 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.027322 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.030889 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.032396 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.032543 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.033180 (+ 637us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.033995 (+ 815us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.034182 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":741,"threads_started":1}
I20260925 09:25:00.038153 22605 tablet_service.cc:1467] Processing CreateTablet for tablet efee3e3d7c99497882f4ee22187d1986 (DEFAULT_TABLE table=test-table [id=736d0fe936254654b0908460d65561cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:00.038014 22475 tablet_service.cc:1467] Processing CreateTablet for tablet efee3e3d7c99497882f4ee22187d1986 (DEFAULT_TABLE table=test-table [id=736d0fe936254654b0908460d65561cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:00.039444 22605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efee3e3d7c99497882f4ee22187d1986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:00.039489 22475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efee3e3d7c99497882f4ee22187d1986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:00.039193 22344 tablet_service.cc:1467] Processing CreateTablet for tablet efee3e3d7c99497882f4ee22187d1986 (DEFAULT_TABLE table=test-table [id=736d0fe936254654b0908460d65561cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:00.040732 22344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efee3e3d7c99497882f4ee22187d1986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:00.053212 22688 tablet_bootstrap.cc:492] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Bootstrap starting.
I20260925 09:25:00.055440 22689 tablet_bootstrap.cc:492] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Bootstrap starting.
I20260925 09:25:00.057531 22690 tablet_bootstrap.cc:492] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Bootstrap starting.
I20260925 09:25:00.058007 22688 tablet_bootstrap.cc:654] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:00.059264 22689 tablet_bootstrap.cc:654] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:00.059458 22688 log.cc:824] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:00.060676 22689 log.cc:824] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:00.061128 22690 tablet_bootstrap.cc:654] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:00.062569 22690 log.cc:824] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:00.064404 22688 tablet_bootstrap.cc:492] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: No bootstrap required, opened a new log
I20260925 09:25:00.064774 22688 ts_tablet_manager.cc:1397] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260925 09:25:00.065428 22690 tablet_bootstrap.cc:492] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: No bootstrap required, opened a new log
I20260925 09:25:00.065734 22690 ts_tablet_manager.cc:1397] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 09:25:00.066624 22689 tablet_bootstrap.cc:492] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: No bootstrap required, opened a new log
I20260925 09:25:00.066905 22689 ts_tablet_manager.cc:1397] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260925 09:25:00.072896 22688 raft_consensus.cc:348] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.073406 22688 raft_consensus.cc:374] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:00.073553 22688 raft_consensus.cc:729] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70fe9e1d76842af9eaa4b239c3f39fd, State: Initialized, Role: FOLLOWER
I20260925 09:25:00.074345 22688 consensus_queue.cc:260] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.075512 22670 heartbeater.cc:503] Master 127.3.82.62:37063 was elected leader, sending a full tablet report...
I20260925 09:25:00.076293 22688 ts_tablet_manager.cc:1428] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260925 09:25:00.077123 22690 raft_consensus.cc:348] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.077602 22690 raft_consensus.cc:374] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:00.077709 22690 raft_consensus.cc:729] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa2916b606374a6a9e5361cd4eeb0fde, State: Initialized, Role: FOLLOWER
I20260925 09:25:00.078117 22689 raft_consensus.cc:348] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.078555 22689 raft_consensus.cc:374] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:00.078405 22690 consensus_queue.cc:260] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.078660 22689 raft_consensus.cc:729] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea846612765641d98bc9d673c49560bd, State: Initialized, Role: FOLLOWER
I20260925 09:25:00.079417 22689 consensus_queue.cc:260] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.080513 22690 ts_tablet_manager.cc:1428] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260925 09:25:00.081521 22540 heartbeater.cc:503] Master 127.3.82.62:37063 was elected leader, sending a full tablet report...
I20260925 09:25:00.082383 22689 ts_tablet_manager.cc:1428] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260925 09:25:00.084718 22696 raft_consensus.cc:482] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:25:00.085207 22696 raft_consensus.cc:504] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.087399 22696 leader_election.cc:288] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f70fe9e1d76842af9eaa4b239c3f39fd (127.3.82.3:46309), fa2916b606374a6a9e5361cd4eeb0fde (127.3.82.1:40817)
I20260925 09:25:00.094682 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.087607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:35563 (local address 127.3.82.3:46309)
0925 09:25:00.087840 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.087849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.089002 (+ 1153us) server_negotiation.cc:407] Connection header received
0925 09:25:00.089269 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.089275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.089352 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.089437 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:00.090305 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.091073 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.092952 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.093730 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.093917 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.094022 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.094302 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.094429 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:25:00.095050 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.087513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:46309 (local address 127.3.82.2:35563)
0925 09:25:00.088897 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:00.088913 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:00.089067 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:00.089703 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:00.089708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:00.089733 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:00.090190 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.090201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.091207 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.091214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.092764 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.092776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.093955 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.093961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.094144 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.094167 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 09:25:00.094276 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 09:25:00.094419 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"mutex_wait_us":3,"thread_start_us":1030,"threads_started":1}
I20260925 09:25:00.095183 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.089340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:57935 (local address 127.3.82.1:40817)
0925 09:25:00.089588 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.089595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.090006 (+ 411us) server_negotiation.cc:407] Connection header received
0925 09:25:00.090186 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.090190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.090258 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.090360 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:00.090933 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.091629 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.093424 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.094281 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.094431 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.094531 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.094813 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.094940 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:25:00.095330 22698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.089148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:40817 (local address 127.3.82.2:57935)
0925 09:25:00.089907 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:00.089930 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:00.090061 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:00.090385 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:00.090390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:00.090415 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:00.090768 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.090775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.091772 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.091777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.093239 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:00.093251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.094468 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.094474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:00.094644 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.094669 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 09:25:00.094779 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 09:25:00.094904 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":395,"threads_started":1}
I20260925 09:25:00.096199 22625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efee3e3d7c99497882f4ee22187d1986" candidate_uuid: "ea846612765641d98bc9d673c49560bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" is_pre_election: true
I20260925 09:25:00.096752 22625 raft_consensus.cc:2454] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea846612765641d98bc9d673c49560bd in term 0.
I20260925 09:25:00.096655 22364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efee3e3d7c99497882f4ee22187d1986" candidate_uuid: "ea846612765641d98bc9d673c49560bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" is_pre_election: true
I20260925 09:25:00.097203 22364 raft_consensus.cc:2454] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea846612765641d98bc9d673c49560bd in term 0.
I20260925 09:25:00.097455 22429 leader_election.cc:302] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea846612765641d98bc9d673c49560bd, f70fe9e1d76842af9eaa4b239c3f39fd; no voters:
I20260925 09:25:00.098001 22696 raft_consensus.cc:2781] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:25:00.098172 22696 raft_consensus.cc:482] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:25:00.098317 22696 raft_consensus.cc:3037] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:00.100848 22696 raft_consensus.cc:504] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.101611 22696 leader_election.cc:288] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [CANDIDATE]: Term 1 election: Requested vote from peers f70fe9e1d76842af9eaa4b239c3f39fd (127.3.82.3:46309), fa2916b606374a6a9e5361cd4eeb0fde (127.3.82.1:40817)
I20260925 09:25:00.102340 22625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efee3e3d7c99497882f4ee22187d1986" candidate_uuid: "ea846612765641d98bc9d673c49560bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd"
I20260925 09:25:00.102411 22364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efee3e3d7c99497882f4ee22187d1986" candidate_uuid: "ea846612765641d98bc9d673c49560bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa2916b606374a6a9e5361cd4eeb0fde"
I20260925 09:25:00.102584 22625 raft_consensus.cc:3037] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:00.102643 22364 raft_consensus.cc:3037] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:00.105085 22625 raft_consensus.cc:2454] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea846612765641d98bc9d673c49560bd in term 1.
I20260925 09:25:00.105118 22364 raft_consensus.cc:2454] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea846612765641d98bc9d673c49560bd in term 1.
I20260925 09:25:00.105643 22429 leader_election.cc:302] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea846612765641d98bc9d673c49560bd, fa2916b606374a6a9e5361cd4eeb0fde; no voters:
I20260925 09:25:00.106014 22696 raft_consensus.cc:2781] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:00.106469 22696 raft_consensus.cc:686] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 1 LEADER]: Becoming Leader. State: Replica: ea846612765641d98bc9d673c49560bd, State: Running, Role: LEADER
I20260925 09:25:00.107034 22696 consensus_queue.cc:237] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } }
I20260925 09:25:00.112003 22223 catalog_manager.cc:5495] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd reported cstate change: term changed from 0 to 1, leader changed from <none> to ea846612765641d98bc9d673c49560bd (127.3.82.2). New cstate: current_term: 1 leader_uuid: "ea846612765641d98bc9d673c49560bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea846612765641d98bc9d673c49560bd" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 40489 } health_report { overall_health: HEALTHY } } }
I20260925 09:25:00.136991 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.130001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.3.82.2:40489)
0925 09:25:00.130287 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.130293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.130315 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:25:00.130549 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.130554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.130657 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.130853 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:00.131278 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.132102 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.133720 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.134296 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.134459 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.135400 (+ 941us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:00.136118 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:00.136374 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.136608 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.136666 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:25:00.146476 22364 raft_consensus.cc:1261] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 1 FOLLOWER]: Refusing update from remote peer ea846612765641d98bc9d673c49560bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:00.146492 22625 raft_consensus.cc:1261] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 1 FOLLOWER]: Refusing update from remote peer ea846612765641d98bc9d673c49560bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:00.147617 22699 consensus_queue.cc:1035] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f70fe9e1d76842af9eaa4b239c3f39fd" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 46309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:00.148106 22696 consensus_queue.cc:1035] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa2916b606374a6a9e5361cd4eeb0fde" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:00.179544 22223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:25:00.179915 22223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:25:00.184036 22223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67465338bd9d44e9804a89891b25a544: Sending DeleteTablet for 3 replicas of tablet efee3e3d7c99497882f4ee22187d1986
I20260925 09:25:00.185513 22475 tablet_service.cc:1514] Processing DeleteTablet for tablet efee3e3d7c99497882f4ee22187d1986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:00 UTC) from {username='slave'} at 127.0.0.1:57970
I20260925 09:25:00.185549 22344 tablet_service.cc:1514] Processing DeleteTablet for tablet efee3e3d7c99497882f4ee22187d1986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:00 UTC) from {username='slave'} at 127.0.0.1:35460
I20260925 09:25:00.185719 22605 tablet_service.cc:1514] Processing DeleteTablet for tablet efee3e3d7c99497882f4ee22187d1986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:00 UTC) from {username='slave'} at 127.0.0.1:57326
I20260925 09:25:00.187033 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22282
I20260925 09:25:00.198552 22714 tablet_replica.cc:330] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: stopping tablet replica
I20260925 09:25:00.199393 22712 tablet_replica.cc:330] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: stopping tablet replica
I20260925 09:25:00.200356 22714 raft_consensus.cc:2229] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:25:00.200943 22714 raft_consensus.cc:2258] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:00.203577 22714 ts_tablet_manager.cc:1905] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:00.200309 22712 raft_consensus.cc:2229] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:25:00.207909 22712 raft_consensus.cc:2258] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:00.211205 22712 ts_tablet_manager.cc:1905] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:00.211584 22714 ts_tablet_manager.cc:1918] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:00.211776 22714 log.cc:1196] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/efee3e3d7c99497882f4ee22187d1986
I20260925 09:25:00.212352 22714 ts_tablet_manager.cc:1939] T efee3e3d7c99497882f4ee22187d1986 P fa2916b606374a6a9e5361cd4eeb0fde: Deleting consensus metadata
I20260925 09:25:00.214587 22208 catalog_manager.cc:4841] TS fa2916b606374a6a9e5361cd4eeb0fde (127.3.82.1:40817): tablet efee3e3d7c99497882f4ee22187d1986 (table test-table [id=736d0fe936254654b0908460d65561cc]) successfully deleted
I20260925 09:25:00.215586 22713 tablet_replica.cc:330] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: stopping tablet replica
I20260925 09:25:00.216099 22713 raft_consensus.cc:2229] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:25:00.216697 22713 raft_consensus.cc:2258] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:00.219612 22713 ts_tablet_manager.cc:1905] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:00.219933 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.189557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.3.82.1:40817)
0925 09:25:00.189778 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.189786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.203225 (+ 13439us) server_negotiation.cc:407] Connection header received
0925 09:25:00.203456 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.203465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.203632 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.203787 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:00.204860 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.206632 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.215451 (+ 8819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.216154 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.216282 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.218055 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:00.218068 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:00.218088 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:00.218940 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:00.219009 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:00.219035 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:00.219129 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.219407 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.219541 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:25:00.221129 22404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:00.222790 22712 ts_tablet_manager.cc:1918] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:00.222992 22712 log.cc:1196] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/efee3e3d7c99497882f4ee22187d1986
I20260925 09:25:00.223609 22712 ts_tablet_manager.cc:1939] T efee3e3d7c99497882f4ee22187d1986 P ea846612765641d98bc9d673c49560bd: Deleting consensus metadata
I20260925 09:25:00.225639 22209 catalog_manager.cc:4841] TS ea846612765641d98bc9d673c49560bd (127.3.82.2:40489): tablet efee3e3d7c99497882f4ee22187d1986 (table test-table [id=736d0fe936254654b0908460d65561cc]) successfully deleted
I20260925 09:25:00.229592 22713 ts_tablet_manager.cc:1918] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:00.229873 22713 log.cc:1196] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/efee3e3d7c99497882f4ee22187d1986
I20260925 09:25:00.230551 22713 ts_tablet_manager.cc:1939] T efee3e3d7c99497882f4ee22187d1986 P f70fe9e1d76842af9eaa4b239c3f39fd: Deleting consensus metadata
I20260925 09:25:00.232851 22208 catalog_manager.cc:4841] TS f70fe9e1d76842af9eaa4b239c3f39fd (127.3.82.3:46309): tablet efee3e3d7c99497882f4ee22187d1986 (table test-table [id=736d0fe936254654b0908460d65561cc]) successfully deleted
I20260925 09:25:00.321715 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22282
I20260925 09:25:00.340371 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22413
I20260925 09:25:00.350243 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.341624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.3.82.2:40489)
0925 09:25:00.341989 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.341999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.342020 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:25:00.342081 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.342085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.342201 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.342307 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:00.342930 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.343865 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.345980 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.346545 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.346938 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.348311 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:00.348323 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:00.348340 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:00.349164 (+ 824us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:00.349223 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:00.349252 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:00.349361 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.349549 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.349650 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 09:25:00.351068 22535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:00.473428 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22413
I20260925 09:25:00.492964 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22543
I20260925 09:25:00.499953 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:00.493956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.3.82.3:46309)
0925 09:25:00.494124 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:00.494129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:00.494145 (+ 16us) server_negotiation.cc:407] Connection header received
0925 09:25:00.494204 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:00.494208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:00.494300 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:00.494412 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:00.494819 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.495587 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.496970 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:00.497360 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:00.497469 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:00.498553 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:00.498561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:00.498577 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:00.499210 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:00.499263 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:00.499283 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:00.499366 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:00.499506 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 09:25:00.499595 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:25:00.500634 22665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:00.598037 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22543
I20260925 09:25:00.617452 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22192
I20260925 09:25:00.618769 22253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:00.732707 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22192
2026-09-25T09:25:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:00 dist-test-slave-4wm5 krb5kdc[22722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:00 dist-test-slave-4wm5 krb5kdc[22722](info): set up 2 sockets
Sep 25 09:25:00 dist-test-slave-4wm5 krb5kdc[22722](info): commencing operation
krb5kdc: starting...
W20260925 09:25:01.890437 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.006s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:02 dist-test-slave-4wm5 krb5kdc[22722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:02.171422 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:35823
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:02.527784 22735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:02.528105 22735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:02.528196 22735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:02.535642 22735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:02.535776 22735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:02.535823 22735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:02.535881 22735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:02.535928 22735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:02.546139 22735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:02.547608 22735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:02.548942 22735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:02.561945 22740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:02.561995 22741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:02.561915 22743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:02.563197 22735 server_base.cc:1034] running on GCE node
I20260925 09:25:02.564076 22735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:02.565560 22735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:02.566776 22735 hybrid_clock.cc:648] HybridClock initialized: now 1790328302566745 us; error 91 us; skew 500 ppm
I20260925 09:25:02.567232 22735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:02 dist-test-slave-4wm5 krb5kdc[22722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:02.570604 22735 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:02.572221 22735 webserver.cc:460] Webserver started at http://127.3.82.62:40117/ using document root <none> and password file <none>
I20260925 09:25:02.572928 22735 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:02.573050 22735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:02.573315 22735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:02.576397 22735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "a47a3b8cdaa64092830b84d787122656"
format_stamp: "Formatted at 2026-09-25 09:25:02 on dist-test-slave-4wm5"
server_key: "6f5632613e83c6165eb6cb52343504e5"
server_key_iv: "6f535abc69a816afa99a1e17058cbbf4"
server_key_version: "encryptionkey@0"
I20260925 09:25:02.577142 22735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "a47a3b8cdaa64092830b84d787122656"
format_stamp: "Formatted at 2026-09-25 09:25:02 on dist-test-slave-4wm5"
server_key: "6f5632613e83c6165eb6cb52343504e5"
server_key_iv: "6f535abc69a816afa99a1e17058cbbf4"
server_key_version: "encryptionkey@0"
I20260925 09:25:02.582702 22735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260925 09:25:02.586422 22750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:02.587890 22735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 09:25:02.588083 22735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "a47a3b8cdaa64092830b84d787122656"
format_stamp: "Formatted at 2026-09-25 09:25:02 on dist-test-slave-4wm5"
server_key: "6f5632613e83c6165eb6cb52343504e5"
server_key_iv: "6f535abc69a816afa99a1e17058cbbf4"
server_key_version: "encryptionkey@0"
I20260925 09:25:02.588281 22735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:02.620271 22735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:02.625140 22735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:02.625411 22735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:02.645370 22735 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:35823
I20260925 09:25:02.645409 22801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:35823 every 8 connection(s)
I20260925 09:25:02.646974 22735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:02.649534 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22735
I20260925 09:25:02.649736 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:02.650282 3400 external_mini_cluster.cc:1428] Setting key 457c184b14a9ec3c749ce1781e1f2ecf
I20260925 09:25:02.651835 22802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:25:02 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:02.667164 22802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656: Bootstrap starting.
I20260925 09:25:02.671114 22802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:02.672286 22802 log.cc:824] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:02.675652 22802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656: No bootstrap required, opened a new log
I20260925 09:25:02.675956 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:02.652563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.3.82.62:35823)
0925 09:25:02.653411 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:02.653423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:02.653465 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:25:02.655209 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:02.655233 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:02.655659 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:02.656038 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:02.658138 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:02.659691 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:02.661601 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:02.662504 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:02.662668 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:02.667842 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:02.667855 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:02.667880 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:02.667930 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:02.671385 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:02.672219 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:02.672235 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:02.672258 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:02.672497 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:02.672931 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:02.672937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:02.672940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:02.673538 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:02.673875 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:02.674401 (+ 526us) server_negotiation.cc:299] Negotiation successful
0925 09:25:02.674863 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":409,"threads_started":1}
I20260925 09:25:02.684512 22802 raft_consensus.cc:348] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47a3b8cdaa64092830b84d787122656" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35823 } }
I20260925 09:25:02.684973 22802 raft_consensus.cc:374] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:02.685087 22802 raft_consensus.cc:729] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47a3b8cdaa64092830b84d787122656, State: Initialized, Role: FOLLOWER
I20260925 09:25:02.685724 22802 consensus_queue.cc:260] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47a3b8cdaa64092830b84d787122656" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35823 } }
I20260925 09:25:02.686030 22802 raft_consensus.cc:388] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:02.686190 22802 raft_consensus.cc:482] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:02.686373 22802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:02.688947 22802 raft_consensus.cc:504] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47a3b8cdaa64092830b84d787122656" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35823 } }
I20260925 09:25:02.689497 22802 leader_election.cc:302] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a47a3b8cdaa64092830b84d787122656; no voters:
I20260925 09:25:02.690152 22802 leader_election.cc:288] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:02.690327 22807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:02.691156 22807 raft_consensus.cc:686] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [term 1 LEADER]: Becoming Leader. State: Replica: a47a3b8cdaa64092830b84d787122656, State: Running, Role: LEADER
I20260925 09:25:02.691903 22807 consensus_queue.cc:237] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47a3b8cdaa64092830b84d787122656" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35823 } }
I20260925 09:25:02.692548 22802 sys_catalog.cc:564] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:02.695466 22809 sys_catalog.cc:455] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a47a3b8cdaa64092830b84d787122656. Latest consensus state: current_term: 1 leader_uuid: "a47a3b8cdaa64092830b84d787122656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47a3b8cdaa64092830b84d787122656" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35823 } } }
I20260925 09:25:02.695364 22808 sys_catalog.cc:455] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a47a3b8cdaa64092830b84d787122656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a47a3b8cdaa64092830b84d787122656" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35823 } } }
I20260925 09:25:02.695835 22808 sys_catalog.cc:458] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:02.695835 22809 sys_catalog.cc:458] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:02.698789 22815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:02.704484 22815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:02.714289 22815 catalog_manager.cc:1347] Generated new cluster ID: 2ed03a8e64994fcd84f7122f39fc1ff6
I20260925 09:25:02.714442 22815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:02.748509 22815 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:02.750653 22815 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:02.763239 22815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656: Generated new TSK 0
I20260925 09:25:02.764483 22815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.82.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
Entry for principal kudu/127.3.82.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab.
I20260925 09:25:02.839210 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35823
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:03.200407 22828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:03.200698 22828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:03.200881 22828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:03.208557 22828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:03.208681 22828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:03.208909 22828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:25:03.219174 22828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.1.keytab
--principal=kudu/127.3.82.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:03.220543 22828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:03.222002 22828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:03.236485 22833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:03.236388 22834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:03.236442 22836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:03.237584 22828 server_base.cc:1034] running on GCE node
I20260925 09:25:03.238369 22828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:03.239607 22828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:03.240990 22828 hybrid_clock.cc:648] HybridClock initialized: now 1790328303240983 us; error 107 us; skew 500 ppm
I20260925 09:25:03.241382 22828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:03 dist-test-slave-4wm5 krb5kdc[22722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328303, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:03.245141 22828 init.cc:377] Logged in from keytab as kudu/127.3.82.1@KRBTEST.COM (short username kudu)
I20260925 09:25:03.246874 22828 webserver.cc:460] Webserver started at http://127.3.82.1:45611/ using document root <none> and password file <none>
I20260925 09:25:03.247699 22828 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:03.247864 22828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:03.248219 22828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:03.251822 22828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "0a30666d68274a78bd321dfe8856e78c"
format_stamp: "Formatted at 2026-09-25 09:25:03 on dist-test-slave-4wm5"
server_key: "368a8bb03cbe12b219eccbafc496b4bf"
server_key_iv: "3013e440b1eb733d819c92014f3b9557"
server_key_version: "encryptionkey@0"
I20260925 09:25:03.252802 22828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "0a30666d68274a78bd321dfe8856e78c"
format_stamp: "Formatted at 2026-09-25 09:25:03 on dist-test-slave-4wm5"
server_key: "368a8bb03cbe12b219eccbafc496b4bf"
server_key_iv: "3013e440b1eb733d819c92014f3b9557"
server_key_version: "encryptionkey@0"
I20260925 09:25:03.259222 22828 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260925 09:25:03.263729 22843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:03.265496 22828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:25:03.265725 22828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "0a30666d68274a78bd321dfe8856e78c"
format_stamp: "Formatted at 2026-09-25 09:25:03 on dist-test-slave-4wm5"
server_key: "368a8bb03cbe12b219eccbafc496b4bf"
server_key_iv: "3013e440b1eb733d819c92014f3b9557"
server_key_version: "encryptionkey@0"
I20260925 09:25:03.266026 22828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:03.288118 22828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:03.293789 22828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:03.294152 22828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:03.295109 22828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:25:03.296525 22828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:25:03.296599 22828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:03.296689 22828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:25:03.296733 22828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:03.333156 22828 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:35593
I20260925 09:25:03.333211 22955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:35593 every 8 connection(s)
I20260925 09:25:03.334712 22828 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:25:03.341346 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22828
I20260925 09:25:03.341571 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:25:03.342191 3400 external_mini_cluster.cc:1428] Setting key 1ca0a19a1694389833c6e185eebc9e95
Sep 25 09:25:03 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328303, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.1@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:03.363292 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:03.337685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:51761 (local address 127.3.82.62:35823)
0925 09:25:03.338246 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:03.338251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:03.339559 (+ 1308us) server_negotiation.cc:407] Connection header received
0925 09:25:03.342366 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:03.342382 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:03.342485 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:03.342589 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:03.344489 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.345273 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.348425 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.348921 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.349047 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:03.354516 (+ 5469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:03.354522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:03.354526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:03.354566 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:03.358900 (+ 4334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:03.359980 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:03.359989 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:03.359995 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:03.360164 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:03.360774 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:03.360782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:03.360788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:03.361230 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:03.361402 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:03.362496 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0925 09:25:03.362789 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":335,"threads_started":1}
I20260925 09:25:03.364109 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:03.338450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35823 (local address 127.3.82.1:51761)
0925 09:25:03.339321 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:03.339388 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:03.341818 (+ 2430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:03.342980 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:03.342991 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:03.343627 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:03.344301 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:03.344320 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.345473 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.345482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:03.348175 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:03.348200 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.349222 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.349233 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:03.349623 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:03.350550 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:03.350598 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:03.354170 (+ 3572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:03.359138 (+ 4968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:03.359148 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:03.359166 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:03.359792 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:03.360365 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:03.360369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:03.360372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:03.360617 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:03.361408 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:03.361417 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:03.362027 (+ 610us) client_negotiation.cc:769] Sending connection context
0925 09:25:03.362337 (+ 310us) client_negotiation.cc:240] Negotiation successful
0925 09:25:03.362823 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":321,"threads_started":1}
I20260925 09:25:03.366124 22956 heartbeater.cc:347] Connected to a master server at 127.3.82.62:35823
I20260925 09:25:03.366487 22956 heartbeater.cc:464] Registering TS with master...
I20260925 09:25:03.367463 22956 heartbeater.cc:511] Master 127.3.82.62:35823 requested a full tablet report, sending...
I20260925 09:25:03.370364 22767 ts_manager.cc:194] Registered new tserver with Master: 0a30666d68274a78bd321dfe8856e78c (127.3.82.1:35593)
I20260925 09:25:03.372136 22767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.1@KRBTEST.COM'} at 127.3.82.1:51761
WARNING: no policy specified for kudu/127.3.82.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
Entry for principal kudu/127.3.82.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab.
I20260925 09:25:03.410449 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35823
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:03.758363 22962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:03.758721 22962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:03.758913 22962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:03.766844 22962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:03.767022 22962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:03.767220 22962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:25:03.778146 22962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.2.keytab
--principal=kudu/127.3.82.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:03.779709 22962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:03.781111 22962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:03.796679 22967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:03.796612 22968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:03.796777 22970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:03.798183 22962 server_base.cc:1034] running on GCE node
I20260925 09:25:03.798893 22962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:03.799980 22962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:03.801198 22962 hybrid_clock.cc:648] HybridClock initialized: now 1790328303801116 us; error 76 us; skew 500 ppm
I20260925 09:25:03.801542 22962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:03 dist-test-slave-4wm5 krb5kdc[22722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328303, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:03.805006 22962 init.cc:377] Logged in from keytab as kudu/127.3.82.2@KRBTEST.COM (short username kudu)
I20260925 09:25:03.806792 22962 webserver.cc:460] Webserver started at http://127.3.82.2:34067/ using document root <none> and password file <none>
I20260925 09:25:03.807564 22962 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:03.807716 22962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:03.808040 22962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:03.811184 22962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "b1783f45b57d42df9d4a2869f1bb69b1"
format_stamp: "Formatted at 2026-09-25 09:25:03 on dist-test-slave-4wm5"
server_key: "923866cdf2d0796b4b09f5e0803c77f7"
server_key_iv: "64a7203b889466b3aa29152fde55c689"
server_key_version: "encryptionkey@0"
I20260925 09:25:03.812397 22962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "b1783f45b57d42df9d4a2869f1bb69b1"
format_stamp: "Formatted at 2026-09-25 09:25:03 on dist-test-slave-4wm5"
server_key: "923866cdf2d0796b4b09f5e0803c77f7"
server_key_iv: "64a7203b889466b3aa29152fde55c689"
server_key_version: "encryptionkey@0"
I20260925 09:25:03.818918 22962 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:25:03.823225 22977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:03.824815 22962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260925 09:25:03.825076 22962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "b1783f45b57d42df9d4a2869f1bb69b1"
format_stamp: "Formatted at 2026-09-25 09:25:03 on dist-test-slave-4wm5"
server_key: "923866cdf2d0796b4b09f5e0803c77f7"
server_key_iv: "64a7203b889466b3aa29152fde55c689"
server_key_version: "encryptionkey@0"
I20260925 09:25:03.825330 22962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:03.844420 22962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:03.850135 22962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:03.850481 22962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:03.851557 22962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:25:03.853063 22962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:25:03.853171 22962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:03.853340 22962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:25:03.853413 22962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:03.890370 22962 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:38717
I20260925 09:25:03.890381 23089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:38717 every 8 connection(s)
I20260925 09:25:03.891782 22962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:25:03.899277 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 22962
I20260925 09:25:03.899546 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:25:03.900045 3400 external_mini_cluster.cc:1428] Setting key b8124ce7d8fa53416123dfcaaa165ddd
Sep 25 09:25:03 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328303, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.2@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:03.916110 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:03.895012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:42217 (local address 127.3.82.62:35823)
0925 09:25:03.895531 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:03.895536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:03.896622 (+ 1086us) server_negotiation.cc:407] Connection header received
0925 09:25:03.898955 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:03.898961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:03.899074 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:03.899231 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:03.901250 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.902272 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.904249 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.904751 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.904918 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:03.909204 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:03.909209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:03.909213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:03.909254 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:03.912432 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:03.913356 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:03.913363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:03.913367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:03.913530 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:03.914122 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:03.914127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:03.914133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:03.914447 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:03.914639 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:03.915424 (+ 785us) server_negotiation.cc:299] Negotiation successful
0925 09:25:03.915679 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":274,"threads_started":1}
I20260925 09:25:03.917227 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:03.895613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35823 (local address 127.3.82.2:42217)
0925 09:25:03.896425 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:03.896474 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:03.898566 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:03.899537 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:03.899551 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:03.900327 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:03.901076 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:03.901102 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.902439 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.902453 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:03.904068 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:03.904082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:03.905034 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:03.905042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:03.905317 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:03.906131 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:03.906159 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:03.908889 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:03.912639 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:03.912651 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:03.912674 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:03.913191 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:03.913666 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:03.913672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:03.913676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:03.913992 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:03.914598 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:03.914608 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:03.915020 (+ 412us) client_negotiation.cc:769] Sending connection context
0925 09:25:03.915326 (+ 306us) client_negotiation.cc:240] Negotiation successful
0925 09:25:03.915943 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":304,"threads_started":1}
I20260925 09:25:03.919019 23090 heartbeater.cc:347] Connected to a master server at 127.3.82.62:35823
I20260925 09:25:03.919366 23090 heartbeater.cc:464] Registering TS with master...
I20260925 09:25:03.920162 23090 heartbeater.cc:511] Master 127.3.82.62:35823 requested a full tablet report, sending...
I20260925 09:25:03.921928 22767 ts_manager.cc:194] Registered new tserver with Master: b1783f45b57d42df9d4a2869f1bb69b1 (127.3.82.2:38717)
I20260925 09:25:03.923321 22767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.2@KRBTEST.COM'} at 127.3.82.2:42217
WARNING: no policy specified for kudu/127.3.82.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
Entry for principal kudu/127.3.82.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab.
I20260925 09:25:03.962916 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35823
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:04.315541 23096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:04.315857 23096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:04.315954 23096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:04.323520 23096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:04.323644 23096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:04.323760 23096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:25:04.334196 23096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.3.keytab
--principal=kudu/127.3.82.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:04.335513 23096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:04.336867 23096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:04.352942 23102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:04.352944 23104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:04.352955 23101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:04.353919 23096 server_base.cc:1034] running on GCE node
I20260925 09:25:04.354566 23096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:04.355593 23096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:04.356803 23096 hybrid_clock.cc:648] HybridClock initialized: now 1790328304356758 us; error 38 us; skew 500 ppm
I20260925 09:25:04.357226 23096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:04 dist-test-slave-4wm5 krb5kdc[22722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:04.360797 23096 init.cc:377] Logged in from keytab as kudu/127.3.82.3@KRBTEST.COM (short username kudu)
I20260925 09:25:04.362514 23096 webserver.cc:460] Webserver started at http://127.3.82.3:46427/ using document root <none> and password file <none>
I20260925 09:25:04.363263 23096 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:04.363374 23096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:04.363631 23096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:04.366743 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "23af2ac7641a4f08bafe8981bd875bd5"
format_stamp: "Formatted at 2026-09-25 09:25:04 on dist-test-slave-4wm5"
server_key: "93ece0bd8111feec801dd739f8e66c0b"
server_key_iv: "99bf396df6f7f32c276128a317412cdc"
server_key_version: "encryptionkey@0"
I20260925 09:25:04.367535 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "23af2ac7641a4f08bafe8981bd875bd5"
format_stamp: "Formatted at 2026-09-25 09:25:04 on dist-test-slave-4wm5"
server_key: "93ece0bd8111feec801dd739f8e66c0b"
server_key_iv: "99bf396df6f7f32c276128a317412cdc"
server_key_version: "encryptionkey@0"
I20260925 09:25:04.373140 23096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:25:04.375777 22956 heartbeater.cc:503] Master 127.3.82.62:35823 was elected leader, sending a full tablet report...
I20260925 09:25:04.378288 23111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:04.379966 23096 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:25:04.380178 23096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "23af2ac7641a4f08bafe8981bd875bd5"
format_stamp: "Formatted at 2026-09-25 09:25:04 on dist-test-slave-4wm5"
server_key: "93ece0bd8111feec801dd739f8e66c0b"
server_key_iv: "99bf396df6f7f32c276128a317412cdc"
server_key_version: "encryptionkey@0"
I20260925 09:25:04.380402 23096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:04.395946 23096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:04.401121 23096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:04.401437 23096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:04.402590 23096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:25:04.403981 23096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:25:04.404054 23096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:04.404206 23096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:25:04.404270 23096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:04.441740 23096 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:38441
I20260925 09:25:04.441962 23223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:38441 every 8 connection(s)
I20260925 09:25:04.443213 23096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:25:04.450894 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23096
I20260925 09:25:04.451123 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:25:04.451653 3400 external_mini_cluster.cc:1428] Setting key b9c6ca97ab3bd4c6aa37fd13d2cc4621
Sep 25 09:25:04 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.3@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:04.471392 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.446570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:41099 (local address 127.3.82.62:35823)
0925 09:25:04.447190 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.447195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.448505 (+ 1310us) server_negotiation.cc:407] Connection header received
0925 09:25:04.450682 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.450689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.450807 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.451189 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:04.453236 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.454376 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.456955 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.457457 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.457581 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.463002 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:04.463012 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:04.463018 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:04.463079 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:04.467669 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:04.468606 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:04.468615 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:04.468618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:04.468761 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:04.469331 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:04.469346 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:04.469349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:04.469644 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:04.469855 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.470693 (+ 838us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.470957 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":358,"threads_started":1}
I20260925 09:25:04.472538 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.447296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35823 (local address 127.3.82.3:41099)
0925 09:25:04.448304 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:04.448349 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:04.450325 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:04.451495 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:04.451506 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:04.452180 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:04.452948 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.452969 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.454604 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.454612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.456732 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.456753 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.457718 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.457724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.458090 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.458967 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:04.458994 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:04.462629 (+ 3635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:04.467862 (+ 5233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:04.467870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:04.467888 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:04.468450 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:04.468972 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:04.468975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:04.468978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:04.469208 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:04.469788 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:04.469797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:04.470283 (+ 486us) client_negotiation.cc:769] Sending connection context
0925 09:25:04.470578 (+ 295us) client_negotiation.cc:240] Negotiation successful
0925 09:25:04.471204 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":468,"threads_started":1}
I20260925 09:25:04.474466 23224 heartbeater.cc:347] Connected to a master server at 127.3.82.62:35823
I20260925 09:25:04.474876 23224 heartbeater.cc:464] Registering TS with master...
I20260925 09:25:04.475705 23224 heartbeater.cc:511] Master 127.3.82.62:35823 requested a full tablet report, sending...
I20260925 09:25:04.477692 22767 ts_manager.cc:194] Registered new tserver with Master: 23af2ac7641a4f08bafe8981bd875bd5 (127.3.82.3:38441)
I20260925 09:25:04.478709 22767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.82.3@KRBTEST.COM'} at 127.3.82.3:41099
I20260925 09:25:04.482156 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:25:04.501538 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.486859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.3.82.62:35823)
0925 09:25:04.487035 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.487039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.487491 (+ 452us) server_negotiation.cc:407] Connection header received
0925 09:25:04.487631 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.487635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.487732 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.487826 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:04.489308 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.490237 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.491878 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.492443 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.492617 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.495649 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:04.495655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:04.495658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:04.495699 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:04.498599 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:04.499368 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:04.499372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:04.499374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:04.499574 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:04.500027 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:04.500031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:04.500032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:04.500366 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:04.500599 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.501059 (+ 460us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.501252 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:25:04.506031 22767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:25:04.513605 22767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:25:04.551367 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.527994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:38717 (local address 127.0.0.1:37018)
0925 09:25:04.540360 (+ 12366us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:04.540434 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:04.540744 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:04.542552 (+ 1808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:04.542559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:04.542598 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:04.543119 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.543134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.545088 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.545094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.547687 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.547702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.550165 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.550172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.550408 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.550490 (+ 82us) client_negotiation.cc:769] Sending connection context
0925 09:25:04.550641 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 09:25:04.550890 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11978,"mutex_wait_us":32,"thread_start_us":14077,"threads_started":1}
I20260925 09:25:04.551455 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.528152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.3.82.2:38717)
0925 09:25:04.541511 (+ 13359us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.541523 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.541562 (+ 39us) server_negotiation.cc:407] Connection header received
0925 09:25:04.541698 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.541706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.542042 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.542262 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:04.543313 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.544913 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.547885 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.549952 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.550093 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.550341 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.550894 (+ 553us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.551089 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13137,"thread_start_us":12977,"threads_started":1}
I20260925 09:25:04.551714 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.528062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:35593 (local address 127.0.0.1:56060)
0925 09:25:04.541916 (+ 13854us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:04.541936 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:04.542915 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:04.544027 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:04.544031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:04.544053 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:04.544479 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.544494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.546941 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.546945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.548777 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.548792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.550831 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.550837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.550995 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.551027 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 09:25:04.551121 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 09:25:04.551261 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13708,"mutex_wait_us":22,"thread_start_us":11736,"threads_started":1}
I20260925 09:25:04.552402 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.528464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.3.82.1:35593)
0925 09:25:04.539286 (+ 10822us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.539301 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.542048 (+ 2747us) server_negotiation.cc:407] Connection header received
0925 09:25:04.543176 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.543187 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.543445 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.543632 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:04.544664 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.546747 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.548946 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.550305 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.550446 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.550661 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.551811 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.552018 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10535,"thread_start_us":10859,"threads_started":1}
I20260925 09:25:04.552716 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.531769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38441 (local address 127.0.0.1:59564)
0925 09:25:04.542914 (+ 11145us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:04.542934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:04.543072 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:04.543747 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:04.543752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:04.543777 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:04.544215 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.544226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.546681 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.546686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.549212 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.549229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.551795 (+ 2566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.551801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.552028 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.552057 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:25:04.552149 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 09:25:04.552296 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10997,"mutex_wait_us":32,"thread_start_us":10804,"threads_started":1}
I20260925 09:25:04.553359 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.532683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.3.82.3:38441)
0925 09:25:04.542201 (+ 9518us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.542214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.543216 (+ 1002us) server_negotiation.cc:407] Connection header received
0925 09:25:04.543338 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.543347 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.543537 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.543729 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:04.544744 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.546461 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.549402 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.551310 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.551433 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.551637 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.552825 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.553008 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9284,"thread_start_us":9058,"threads_started":1}
I20260925 09:25:04.556542 23025 tablet_service.cc:1467] Processing CreateTablet for tablet d2cf79a46c2a44b28daab292983b9dbf (DEFAULT_TABLE table=test-table [id=b365f0d3813b4aa9a8ed1c60bb8cd584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:04.556644 22891 tablet_service.cc:1467] Processing CreateTablet for tablet d2cf79a46c2a44b28daab292983b9dbf (DEFAULT_TABLE table=test-table [id=b365f0d3813b4aa9a8ed1c60bb8cd584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:04.557996 23025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2cf79a46c2a44b28daab292983b9dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:04.557432 23159 tablet_service.cc:1467] Processing CreateTablet for tablet d2cf79a46c2a44b28daab292983b9dbf (DEFAULT_TABLE table=test-table [id=b365f0d3813b4aa9a8ed1c60bb8cd584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:04.558251 22891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2cf79a46c2a44b28daab292983b9dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:04.558902 23159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2cf79a46c2a44b28daab292983b9dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:04.570003 23243 tablet_bootstrap.cc:492] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Bootstrap starting.
I20260925 09:25:04.574051 23243 tablet_bootstrap.cc:654] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:04.575438 23245 tablet_bootstrap.cc:492] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Bootstrap starting.
I20260925 09:25:04.575557 23243 log.cc:824] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:04.576442 23244 tablet_bootstrap.cc:492] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Bootstrap starting.
I20260925 09:25:04.579341 23243 tablet_bootstrap.cc:492] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: No bootstrap required, opened a new log
I20260925 09:25:04.579643 23243 ts_tablet_manager.cc:1397] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260925 09:25:04.580444 23245 tablet_bootstrap.cc:654] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:04.580492 23244 tablet_bootstrap.cc:654] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:04.581990 23245 log.cc:824] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:04.582018 23244 log.cc:824] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:04.586402 23244 tablet_bootstrap.cc:492] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: No bootstrap required, opened a new log
I20260925 09:25:04.586421 23245 tablet_bootstrap.cc:492] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: No bootstrap required, opened a new log
I20260925 09:25:04.586714 23244 ts_tablet_manager.cc:1397] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260925 09:25:04.586758 23245 ts_tablet_manager.cc:1397] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260925 09:25:04.591814 23243 raft_consensus.cc:348] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.592482 23243 raft_consensus.cc:374] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:04.592686 23243 raft_consensus.cc:729] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1783f45b57d42df9d4a2869f1bb69b1, State: Initialized, Role: FOLLOWER
I20260925 09:25:04.593730 23243 consensus_queue.cc:260] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.595575 23090 heartbeater.cc:503] Master 127.3.82.62:35823 was elected leader, sending a full tablet report...
I20260925 09:25:04.595839 23244 raft_consensus.cc:348] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.596321 23244 raft_consensus.cc:374] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:04.596417 23244 raft_consensus.cc:729] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23af2ac7641a4f08bafe8981bd875bd5, State: Initialized, Role: FOLLOWER
I20260925 09:25:04.596433 23243 ts_tablet_manager.cc:1428] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260925 09:25:04.597272 23244 consensus_queue.cc:260] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.598582 23224 heartbeater.cc:503] Master 127.3.82.62:35823 was elected leader, sending a full tablet report...
I20260925 09:25:04.599303 23244 ts_tablet_manager.cc:1428] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Time spent starting tablet: real 0.012s user 0.003s sys 0.009s
I20260925 09:25:04.600862 23245 raft_consensus.cc:348] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.601392 23245 raft_consensus.cc:374] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:04.601540 23245 raft_consensus.cc:729] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a30666d68274a78bd321dfe8856e78c, State: Initialized, Role: FOLLOWER
I20260925 09:25:04.602310 23245 consensus_queue.cc:260] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.604559 23245 ts_tablet_manager.cc:1428] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
W20260925 09:25:04.644073 23091 tablet.cc:2369] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:25:04.695094 23225 tablet.cc:2369] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:25:04.832513 23251 raft_consensus.cc:482] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:25:04.832911 23251 raft_consensus.cc:504] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.835211 23251 leader_election.cc:288] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1783f45b57d42df9d4a2869f1bb69b1 (127.3.82.2:38717), 23af2ac7641a4f08bafe8981bd875bd5 (127.3.82.3:38441)
W20260925 09:25:04.837947 22957 tablet.cc:2369] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:25:04.845077 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.836230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:47931 (local address 127.3.82.2:38717)
0925 09:25:04.836451 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.836458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.836754 (+ 296us) server_negotiation.cc:407] Connection header received
0925 09:25:04.837040 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.837046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.837142 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.837248 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:04.838237 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.839481 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.842772 (+ 3291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.844035 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.844206 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.844307 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.844647 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.844841 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:25:04.845189 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.835593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:38717 (local address 127.3.82.1:47931)
0925 09:25:04.836610 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:04.836637 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:04.836892 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:04.837265 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:04.837269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:04.837304 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:04.838033 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.838063 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.840034 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.840039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.842540 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.842558 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.844275 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.844281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.844472 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.844509 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 09:25:04.844610 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 09:25:04.844758 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":26,"thread_start_us":1093,"threads_started":1}
I20260925 09:25:04.846460 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.835607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:38441 (local address 127.3.82.1:50067)
0925 09:25:04.837073 (+ 1466us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:04.837108 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:04.837256 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:04.839346 (+ 2090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:04.839353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:04.839446 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:04.839855 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.839870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.841449 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.841453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.843732 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:04.843750 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.845631 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.845636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:04.845842 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.845910 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 09:25:04.846051 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 09:25:04.846178 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":30,"thread_start_us":560,"threads_started":1}
I20260925 09:25:04.846516 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.837544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:50067 (local address 127.3.82.3:38441)
0925 09:25:04.837884 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.837890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.837911 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:25:04.839085 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.839091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.839177 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.839274 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:04.840109 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.841271 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.843993 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.845238 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.845380 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.845485 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.846098 (+ 613us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.846259 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 09:25:04.846599 23045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2cf79a46c2a44b28daab292983b9dbf" candidate_uuid: "0a30666d68274a78bd321dfe8856e78c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" is_pre_election: true
I20260925 09:25:04.847225 23045 raft_consensus.cc:2454] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a30666d68274a78bd321dfe8856e78c in term 0.
I20260925 09:25:04.847601 23179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2cf79a46c2a44b28daab292983b9dbf" candidate_uuid: "0a30666d68274a78bd321dfe8856e78c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23af2ac7641a4f08bafe8981bd875bd5" is_pre_election: true
I20260925 09:25:04.848173 23179 raft_consensus.cc:2454] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a30666d68274a78bd321dfe8856e78c in term 0.
I20260925 09:25:04.848136 22846 leader_election.cc:302] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30666d68274a78bd321dfe8856e78c, b1783f45b57d42df9d4a2869f1bb69b1; no voters:
I20260925 09:25:04.848660 23251 raft_consensus.cc:2781] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:25:04.848837 23251 raft_consensus.cc:482] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:25:04.848968 23251 raft_consensus.cc:3037] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:04.852644 23251 raft_consensus.cc:504] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.853447 23251 leader_election.cc:288] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [CANDIDATE]: Term 1 election: Requested vote from peers b1783f45b57d42df9d4a2869f1bb69b1 (127.3.82.2:38717), 23af2ac7641a4f08bafe8981bd875bd5 (127.3.82.3:38441)
I20260925 09:25:04.854084 23045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2cf79a46c2a44b28daab292983b9dbf" candidate_uuid: "0a30666d68274a78bd321dfe8856e78c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1783f45b57d42df9d4a2869f1bb69b1"
I20260925 09:25:04.854344 23045 raft_consensus.cc:3037] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:04.854358 23179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2cf79a46c2a44b28daab292983b9dbf" candidate_uuid: "0a30666d68274a78bd321dfe8856e78c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23af2ac7641a4f08bafe8981bd875bd5"
I20260925 09:25:04.854552 23179 raft_consensus.cc:3037] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:04.856737 23045 raft_consensus.cc:2454] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a30666d68274a78bd321dfe8856e78c in term 1.
I20260925 09:25:04.856737 23179 raft_consensus.cc:2454] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a30666d68274a78bd321dfe8856e78c in term 1.
I20260925 09:25:04.857309 22846 leader_election.cc:302] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a30666d68274a78bd321dfe8856e78c, b1783f45b57d42df9d4a2869f1bb69b1; no voters:
I20260925 09:25:04.857699 23251 raft_consensus.cc:2781] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:04.858300 23251 raft_consensus.cc:686] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 1 LEADER]: Becoming Leader. State: Replica: 0a30666d68274a78bd321dfe8856e78c, State: Running, Role: LEADER
I20260925 09:25:04.858955 23251 consensus_queue.cc:237] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } }
I20260925 09:25:04.864995 22767 catalog_manager.cc:5495] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a30666d68274a78bd321dfe8856e78c (127.3.82.1). New cstate: current_term: 1 leader_uuid: "0a30666d68274a78bd321dfe8856e78c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a30666d68274a78bd321dfe8856e78c" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 35593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 } health_report { overall_health: UNKNOWN } } }
I20260925 09:25:04.925586 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.919776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56070 (local address 127.3.82.1:35593)
0925 09:25:04.920026 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.920035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.920060 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:25:04.920379 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.920383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.920464 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.920626 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:04.921010 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.921742 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.923342 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:04.923754 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:04.923893 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:04.924452 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:04.924962 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:04.925125 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:04.925224 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 09:25:04.925319 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260925 09:25:04.935352 23045 raft_consensus.cc:1261] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 1 FOLLOWER]: Refusing update from remote peer 0a30666d68274a78bd321dfe8856e78c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:04.935345 23179 raft_consensus.cc:1261] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 1 FOLLOWER]: Refusing update from remote peer 0a30666d68274a78bd321dfe8856e78c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:04.936672 23251 consensus_queue.cc:1035] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23af2ac7641a4f08bafe8981bd875bd5" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:04.937162 23254 consensus_queue.cc:1035] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1783f45b57d42df9d4a2869f1bb69b1" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 38717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:04.952768 23259 mvcc.cc:205] Tried to move back new op lower bound from 7333184737002622976 to 7333184736707760128. Current Snapshot: MvccSnapshot[applied={T|T < 7333184737002622976}]
I20260925 09:25:04.954620 23258 mvcc.cc:205] Tried to move back new op lower bound from 7333184737002622976 to 7333184736707760128. Current Snapshot: MvccSnapshot[applied={T|T < 7333184737002622976}]
I20260925 09:25:04.967355 22766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:25:04.967685 22766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:25:04.972311 22766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a47a3b8cdaa64092830b84d787122656: Sending DeleteTablet for 3 replicas of tablet d2cf79a46c2a44b28daab292983b9dbf
I20260925 09:25:04.973685 22891 tablet_service.cc:1514] Processing DeleteTablet for tablet d2cf79a46c2a44b28daab292983b9dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:04 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:56060
I20260925 09:25:04.973896 23025 tablet_service.cc:1514] Processing DeleteTablet for tablet d2cf79a46c2a44b28daab292983b9dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:04 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:37018
I20260925 09:25:04.974802 23266 tablet_replica.cc:330] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: stopping tablet replica
I20260925 09:25:04.974797 23159 tablet_service.cc:1514] Processing DeleteTablet for tablet d2cf79a46c2a44b28daab292983b9dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:04 UTC) from {username='kudu', principal='kudu/127.3.82.62@KRBTEST.COM'} at 127.0.0.1:59564
I20260925 09:25:04.975355 23266 raft_consensus.cc:2229] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:25:04.976190 23266 raft_consensus.cc:2258] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:04.976759 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22828
I20260925 09:25:04.978786 23266 ts_tablet_manager.cc:1905] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:04.989558 23267 tablet_replica.cc:330] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: stopping tablet replica
I20260925 09:25:04.989521 23266 ts_tablet_manager.cc:1918] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:04.989773 23266 log.cc:1196] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/d2cf79a46c2a44b28daab292983b9dbf
I20260925 09:25:04.990258 23267 raft_consensus.cc:2229] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:25:04.990600 23266 ts_tablet_manager.cc:1939] T d2cf79a46c2a44b28daab292983b9dbf P 0a30666d68274a78bd321dfe8856e78c: Deleting consensus metadata
I20260925 09:25:04.990870 23267 raft_consensus.cc:2258] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:04.993274 22752 catalog_manager.cc:4841] TS 0a30666d68274a78bd321dfe8856e78c (127.3.82.1:35593): tablet d2cf79a46c2a44b28daab292983b9dbf (table test-table [id=b365f0d3813b4aa9a8ed1c60bb8cd584]) successfully deleted
I20260925 09:25:04.993484 23267 ts_tablet_manager.cc:1905] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:04.999966 23268 tablet_replica.cc:330] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: stopping tablet replica
I20260925 09:25:05.000633 23268 raft_consensus.cc:2229] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:25:05.001144 23267 ts_tablet_manager.cc:1918] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:05.001381 23268 raft_consensus.cc:2258] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:05.001405 23267 log.cc:1196] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/d2cf79a46c2a44b28daab292983b9dbf
I20260925 09:25:05.002405 23267 ts_tablet_manager.cc:1939] T d2cf79a46c2a44b28daab292983b9dbf P b1783f45b57d42df9d4a2869f1bb69b1: Deleting consensus metadata
I20260925 09:25:05.004235 23268 ts_tablet_manager.cc:1905] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:05.004630 22753 catalog_manager.cc:4841] TS b1783f45b57d42df9d4a2869f1bb69b1 (127.3.82.2:38717): tablet d2cf79a46c2a44b28daab292983b9dbf (table test-table [id=b365f0d3813b4aa9a8ed1c60bb8cd584]) successfully deleted
I20260925 09:25:05.011070 23268 ts_tablet_manager.cc:1918] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:05.011240 23268 log.cc:1196] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/d2cf79a46c2a44b28daab292983b9dbf
I20260925 09:25:05.011827 23268 ts_tablet_manager.cc:1939] T d2cf79a46c2a44b28daab292983b9dbf P 23af2ac7641a4f08bafe8981bd875bd5: Deleting consensus metadata
I20260925 09:25:05.013751 22754 catalog_manager.cc:4841] TS 23af2ac7641a4f08bafe8981bd875bd5 (127.3.82.3:38441): tablet d2cf79a46c2a44b28daab292983b9dbf (table test-table [id=b365f0d3813b4aa9a8ed1c60bb8cd584]) successfully deleted
Sep 25 09:25:05 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.1@KRBTEST.COM
I20260925 09:25:05.026322 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:04.980201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.3.82.1:35593)
0925 09:25:04.980743 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:04.980750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:04.999130 (+ 18380us) server_negotiation.cc:407] Connection header received
0925 09:25:04.999373 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:04.999382 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:04.999497 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:04.999614 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:05.001868 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:05.003335 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:05.005568 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:05.006412 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:05.006575 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:05.018573 (+ 11998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:05.018590 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:05.018610 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:05.018682 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:05.022998 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:05.023902 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:05.023918 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:05.023922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:05.024109 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:05.024535 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:05.024540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:05.024543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:05.025013 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:05.025352 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:05.025598 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 09:25:05.025914 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397}
I20260925 09:25:05.027256 22951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:05.134136 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22828
I20260925 09:25:05.155761 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22962
Sep 25 09:25:05 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.2@KRBTEST.COM
I20260925 09:25:05.173493 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:05.156859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.3.82.2:38717)
0925 09:25:05.157062 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:05.157067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:05.157087 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:25:05.157228 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:05.157232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:05.157299 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:05.157395 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:05.158996 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:05.159750 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:05.161167 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:05.161606 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:05.161709 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:05.166661 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:05.166679 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:05.166695 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:05.166738 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:05.170191 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:05.171047 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:05.171055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:05.171058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:05.171211 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:05.171634 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:05.171638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:05.171640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:05.172109 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:05.172344 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:05.172804 (+ 460us) server_negotiation.cc:299] Negotiation successful
0925 09:25:05.173010 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 09:25:05.174461 23085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:05.275709 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22962
I20260925 09:25:05.296152 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23096
Sep 25 09:25:05 dist-test-slave-4wm5 krb5kdc[22722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.3@KRBTEST.COM
I20260925 09:25:05.313690 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:05.297116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.3.82.3:38441)
0925 09:25:05.297288 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:05.297293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:05.297312 (+ 19us) server_negotiation.cc:407] Connection header received
0925 09:25:05.297400 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:05.297404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:05.297470 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:05.297563 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:05.299101 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:05.299902 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:05.301312 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:05.301704 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:05.301805 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:05.307060 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:05.307077 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:05.307133 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:05.307196 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:05.310644 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:05.311517 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:05.311524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:05.311527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:05.311679 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:05.312125 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:05.312129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:05.312131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:05.312539 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:05.312858 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:05.313159 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 09:25:05.313345 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:25:05.314595 23219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:05.408998 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23096
I20260925 09:25:05.425876 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 22735
I20260925 09:25:05.427058 22797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:05.543294 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 22735
2026-09-25T09:25:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4811 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T09:25:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:05Z Disabled control of system clock
I20260925 09:25:05.625708 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:38611
--webserver_interface=127.3.82.62
--webserver_port=0
--builtin_ntp_servers=127.3.82.20:33753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:38611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:25:05.985653 23277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:05.985992 23277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:05.986076 23277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:05.993559 23277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:05.993695 23277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:05.993742 23277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:05.993796 23277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:05.993917 23277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:06.003774 23277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:38611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:38611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:06.005060 23277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:06.006932 23277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:06.019894 23282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:06.019851 23283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:06.019933 23285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:06.020746 23277 server_base.cc:1034] running on GCE node
I20260925 09:25:06.021793 23277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:06.023502 23277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:06.024843 23277 hybrid_clock.cc:648] HybridClock initialized: now 1790328306024786 us; error 51 us; skew 500 ppm
I20260925 09:25:06.025286 23277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:25:06.028162 23277 webserver.cc:460] Webserver started at http://127.3.82.62:44265/ using document root <none> and password file <none>
I20260925 09:25:06.028978 23277 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:06.029110 23277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:06.029511 23277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:06.032996 23277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "215143838aaa49828c5eed4284fb3cac"
format_stamp: "Formatted at 2026-09-25 09:25:06 on dist-test-slave-4wm5"
server_key: "c26c23b88ff154e7df55280abc5dc126"
server_key_iv: "7d137a138025eee81600d66b78d6379a"
server_key_version: "encryptionkey@0"
I20260925 09:25:06.034037 23277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "215143838aaa49828c5eed4284fb3cac"
format_stamp: "Formatted at 2026-09-25 09:25:06 on dist-test-slave-4wm5"
server_key: "c26c23b88ff154e7df55280abc5dc126"
server_key_iv: "7d137a138025eee81600d66b78d6379a"
server_key_version: "encryptionkey@0"
I20260925 09:25:06.039875 23277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:25:06.044348 23291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:06.046185 23277 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:06.046412 23277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "215143838aaa49828c5eed4284fb3cac"
format_stamp: "Formatted at 2026-09-25 09:25:06 on dist-test-slave-4wm5"
server_key: "c26c23b88ff154e7df55280abc5dc126"
server_key_iv: "7d137a138025eee81600d66b78d6379a"
server_key_version: "encryptionkey@0"
I20260925 09:25:06.046681 23277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:06.066164 23277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:06.067358 23277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:06.067835 23277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:06.089324 23277 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:38611
I20260925 09:25:06.089347 23342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:38611 every 8 connection(s)
I20260925 09:25:06.090945 23277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:06.093219 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23277
I20260925 09:25:06.093467 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:06.094033 3400 external_mini_cluster.cc:1428] Setting key e8460992a5db7ecdf57f02209677eb0c
I20260925 09:25:06.095504 23343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:06.110010 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:06.096176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.3.82.62:38611)
0925 09:25:06.097319 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:06.097336 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:06.097388 (+ 52us) server_negotiation.cc:407] Connection header received
0925 09:25:06.099181 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:06.099208 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:06.099848 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:06.100320 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:06.100869 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:06.102957 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:06.104435 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:06.105463 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:06.105654 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:06.106920 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:06.106931 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:06.106956 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:06.107773 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:06.107880 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:06.107912 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:06.108040 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:06.108270 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 09:25:06.108653 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":351,"threads_started":1}
I20260925 09:25:06.111045 23343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac: Bootstrap starting.
I20260925 09:25:06.114413 23343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:06.115496 23343 log.cc:824] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:06.119022 23343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac: No bootstrap required, opened a new log
I20260925 09:25:06.126888 23343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215143838aaa49828c5eed4284fb3cac" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 38611 } }
I20260925 09:25:06.127355 23343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:06.127502 23343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215143838aaa49828c5eed4284fb3cac, State: Initialized, Role: FOLLOWER
I20260925 09:25:06.128209 23343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215143838aaa49828c5eed4284fb3cac" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 38611 } }
I20260925 09:25:06.128459 23343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:06.128603 23343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:06.128750 23343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:06.131490 23343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215143838aaa49828c5eed4284fb3cac" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 38611 } }
I20260925 09:25:06.132030 23343 leader_election.cc:302] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 215143838aaa49828c5eed4284fb3cac; no voters:
I20260925 09:25:06.132639 23343 leader_election.cc:288] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:06.132964 23348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:06.133754 23348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [term 1 LEADER]: Becoming Leader. State: Replica: 215143838aaa49828c5eed4284fb3cac, State: Running, Role: LEADER
I20260925 09:25:06.134419 23348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215143838aaa49828c5eed4284fb3cac" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 38611 } }
I20260925 09:25:06.134971 23343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:06.137267 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "215143838aaa49828c5eed4284fb3cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215143838aaa49828c5eed4284fb3cac" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 38611 } } }
I20260925 09:25:06.137703 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:06.140130 23350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215143838aaa49828c5eed4284fb3cac. Latest consensus state: current_term: 1 leader_uuid: "215143838aaa49828c5eed4284fb3cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215143838aaa49828c5eed4284fb3cac" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 38611 } } }
I20260925 09:25:06.140547 23350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:06.142112 23354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:06.149001 23354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:06.161137 23354 catalog_manager.cc:1347] Generated new cluster ID: c95614dfe48d4ec78a74e98fd9e3d4e0
I20260925 09:25:06.161293 23354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:06.174679 23354 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:06.176642 23354 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:06.187282 23354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac: Generated new TSK 0
I20260925 09:25:06.188481 23354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:06.200249 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.1:0
--local_ip_for_outbound_sockets=127.3.82.1
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:38611
--builtin_ntp_servers=127.3.82.20:33753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 09:25:06.555683 23367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:06.556015 23367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:06.556108 23367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:06.563676 23367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:06.563802 23367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:06.563912 23367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.1
I20260925 09:25:06.573405 23367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.82.1
--webserver_port=0
--tserver_master_addrs=127.3.82.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.1
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:06.574730 23367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:06.576570 23367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:06.590441 23372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:06.590505 23373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:06.590519 23375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:06.591432 23367 server_base.cc:1034] running on GCE node
I20260925 09:25:06.592163 23367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:06.593289 23367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:06.594547 23367 hybrid_clock.cc:648] HybridClock initialized: now 1790328306594486 us; error 51 us; skew 500 ppm
I20260925 09:25:06.594983 23367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:25:06.597476 23367 webserver.cc:460] Webserver started at http://127.3.82.1:32865/ using document root <none> and password file <none>
I20260925 09:25:06.598366 23367 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:06.598506 23367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:06.598853 23367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:06.601922 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data/instance:
uuid: "949f38a9aa954ff3bceb46bff0f9b755"
format_stamp: "Formatted at 2026-09-25 09:25:06 on dist-test-slave-4wm5"
server_key: "fe09b9aaad1ed06a1c48863c1f4101b4"
server_key_iv: "245ea26dfd0c83c85a4430153ef6bcdd"
server_key_version: "encryptionkey@0"
I20260925 09:25:06.602784 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance:
uuid: "949f38a9aa954ff3bceb46bff0f9b755"
format_stamp: "Formatted at 2026-09-25 09:25:06 on dist-test-slave-4wm5"
server_key: "fe09b9aaad1ed06a1c48863c1f4101b4"
server_key_iv: "245ea26dfd0c83c85a4430153ef6bcdd"
server_key_version: "encryptionkey@0"
I20260925 09:25:06.608331 23367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 09:25:06.612385 23381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:06.614266 23367 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:06.614441 23367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal
uuid: "949f38a9aa954ff3bceb46bff0f9b755"
format_stamp: "Formatted at 2026-09-25 09:25:06 on dist-test-slave-4wm5"
server_key: "fe09b9aaad1ed06a1c48863c1f4101b4"
server_key_iv: "245ea26dfd0c83c85a4430153ef6bcdd"
server_key_version: "encryptionkey@0"
I20260925 09:25:06.614686 23367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:06.663359 23367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:06.664326 23367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:06.664587 23367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:06.665635 23367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:25:06.667466 23367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:25:06.667564 23367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:06.667649 23367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:25:06.667706 23367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:06.703222 23367 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.1:34705
I20260925 09:25:06.703321 23493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.1:34705 every 8 connection(s)
I20260925 09:25:06.704577 23367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/data/info.pb
I20260925 09:25:06.709206 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23367
I20260925 09:25:06.709417 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal/instance
I20260925 09:25:06.710206 3400 external_mini_cluster.cc:1428] Setting key d42393808734fa403662ac16356b2b9e
I20260925 09:25:06.713757 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.2:0
--local_ip_for_outbound_sockets=127.3.82.2
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:38611
--builtin_ntp_servers=127.3.82.20:33753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:25:06.720798 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:06.707673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.1:37569 (local address 127.3.82.62:38611)
0925 09:25:06.708157 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:06.708161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:06.709693 (+ 1532us) server_negotiation.cc:407] Connection header received
0925 09:25:06.712643 (+ 2950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:06.712650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:06.712778 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:06.712926 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:06.714394 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:06.715236 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:06.717375 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:06.717950 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:06.718117 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:06.719862 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:06.719867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:06.719870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:06.719891 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:06.719931 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:06.719943 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:06.720052 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:06.720307 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 09:25:06.720426 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":266,"threads_started":1}
I20260925 09:25:06.722126 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:06.708107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:38611 (local address 127.3.82.1:37569)
0925 09:25:06.709471 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:06.709536 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:06.712278 (+ 2742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:06.713224 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:06.713234 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:06.713344 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:06.714177 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:06.714208 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:06.715450 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:06.715457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:06.717162 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:06.717179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:06.718272 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:06.718283 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:06.718576 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:06.719421 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:25:06.719424 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:06.719468 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:25:06.719567 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:06.720053 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:06.720067 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:06.720077 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 09:25:06.720286 (+ 209us) client_negotiation.cc:240] Negotiation successful
0925 09:25:06.720739 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":340,"threads_started":1}
I20260925 09:25:06.723992 23494 heartbeater.cc:347] Connected to a master server at 127.3.82.62:38611
I20260925 09:25:06.724339 23494 heartbeater.cc:464] Registering TS with master...
I20260925 09:25:06.725157 23494 heartbeater.cc:511] Master 127.3.82.62:38611 requested a full tablet report, sending...
I20260925 09:25:06.727636 23308 ts_manager.cc:194] Registered new tserver with Master: 949f38a9aa954ff3bceb46bff0f9b755 (127.3.82.1:34705)
I20260925 09:25:06.729166 23308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.1:37569
W20260925 09:25:07.070242 23498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:07.070552 23498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:07.070745 23498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:07.078691 23498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:07.078821 23498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:07.078997 23498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.2
I20260925 09:25:07.090126 23498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.82.2
--webserver_port=0
--tserver_master_addrs=127.3.82.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.2
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:07.091951 23498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:07.094404 23498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:07.109701 23503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:07.110982 23504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:07.111703 23506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:07.113130 23498 server_base.cc:1034] running on GCE node
I20260925 09:25:07.113906 23498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:07.114959 23498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:07.116289 23498 hybrid_clock.cc:648] HybridClock initialized: now 1790328307116219 us; error 94 us; skew 500 ppm
I20260925 09:25:07.116730 23498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:25:07.119663 23498 webserver.cc:460] Webserver started at http://127.3.82.2:44879/ using document root <none> and password file <none>
I20260925 09:25:07.120404 23498 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:07.120574 23498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:07.120903 23498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:07.123965 23498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data/instance:
uuid: "906e7d262830442caa118357343f5aba"
format_stamp: "Formatted at 2026-09-25 09:25:07 on dist-test-slave-4wm5"
server_key: "204bf17143d791475ac2bdec37b28a44"
server_key_iv: "36c8361ce44807c38fff5288e7f7d9c1"
server_key_version: "encryptionkey@0"
I20260925 09:25:07.124833 23498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance:
uuid: "906e7d262830442caa118357343f5aba"
format_stamp: "Formatted at 2026-09-25 09:25:07 on dist-test-slave-4wm5"
server_key: "204bf17143d791475ac2bdec37b28a44"
server_key_iv: "36c8361ce44807c38fff5288e7f7d9c1"
server_key_version: "encryptionkey@0"
I20260925 09:25:07.130482 23498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:25:07.134341 23512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:07.136025 23498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 09:25:07.136222 23498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal
uuid: "906e7d262830442caa118357343f5aba"
format_stamp: "Formatted at 2026-09-25 09:25:07 on dist-test-slave-4wm5"
server_key: "204bf17143d791475ac2bdec37b28a44"
server_key_iv: "36c8361ce44807c38fff5288e7f7d9c1"
server_key_version: "encryptionkey@0"
I20260925 09:25:07.136535 23498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:07.163637 23498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:07.164523 23498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:07.164784 23498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:07.165773 23498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:25:07.167408 23498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:25:07.167517 23498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:07.167593 23498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:25:07.167646 23498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:07.204139 23498 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.2:45687
I20260925 09:25:07.204237 23624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.2:45687 every 8 connection(s)
I20260925 09:25:07.205480 23498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/data/info.pb
I20260925 09:25:07.213485 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23498
I20260925 09:25:07.213670 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal/instance
I20260925 09:25:07.214167 3400 external_mini_cluster.cc:1428] Setting key 0a61db5b69fdbb6d70e897c61d98a06e
I20260925 09:25:07.217384 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.82.3:0
--local_ip_for_outbound_sockets=127.3.82.3
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:38611
--builtin_ntp_servers=127.3.82.20:33753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 09:25:07.220479 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.208652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:57959 (local address 127.3.82.62:38611)
0925 09:25:07.208902 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:07.208906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:07.210451 (+ 1545us) server_negotiation.cc:407] Connection header received
0925 09:25:07.212264 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:07.212269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:07.212391 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:07.212513 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:07.213762 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.214613 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.216947 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.217448 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.217674 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.219638 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:07.219643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:07.219647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:07.219666 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:07.219723 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:07.219736 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:07.219883 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:07.220143 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 09:25:07.220226 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260925 09:25:07.222072 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.209198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:38611 (local address 127.3.82.2:57959)
0925 09:25:07.210267 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:07.210314 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:07.211983 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:07.212800 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:07.212812 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:07.212928 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:07.213535 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.213579 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.214808 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.214812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.216654 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.216669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.217889 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.217899 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.218298 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.219232 (+ 934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:25:07.219237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:07.219298 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:25:07.219423 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:07.219902 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:07.219916 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:07.219928 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 09:25:07.220131 (+ 203us) client_negotiation.cc:240] Negotiation successful
0925 09:25:07.220593 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":310,"threads_started":1}
I20260925 09:25:07.223869 23625 heartbeater.cc:347] Connected to a master server at 127.3.82.62:38611
I20260925 09:25:07.224218 23625 heartbeater.cc:464] Registering TS with master...
I20260925 09:25:07.225072 23625 heartbeater.cc:511] Master 127.3.82.62:38611 requested a full tablet report, sending...
I20260925 09:25:07.226819 23308 ts_manager.cc:194] Registered new tserver with Master: 906e7d262830442caa118357343f5aba (127.3.82.2:45687)
I20260925 09:25:07.227705 23308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.2:57959
W20260925 09:25:07.565492 23628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:07.565944 23628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:07.566108 23628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:07.573611 23628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:07.573779 23628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:07.573987 23628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.82.3
I20260925 09:25:07.583655 23628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.82.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.82.3
--webserver_port=0
--tserver_master_addrs=127.3.82.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.82.3
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:07.585086 23628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:07.587185 23628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:07.601475 23636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:07.601526 23634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:07.601426 23633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:07.602659 23628 server_base.cc:1034] running on GCE node
I20260925 09:25:07.603286 23628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:07.604326 23628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:07.605538 23628 hybrid_clock.cc:648] HybridClock initialized: now 1790328307605478 us; error 53 us; skew 500 ppm
I20260925 09:25:07.606068 23628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:25:07.608537 23628 webserver.cc:460] Webserver started at http://127.3.82.3:46263/ using document root <none> and password file <none>
I20260925 09:25:07.609300 23628 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:07.609418 23628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:07.609879 23628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:07.614883 23628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data/instance:
uuid: "b60b56d702634dd88265b684f88c8283"
format_stamp: "Formatted at 2026-09-25 09:25:07 on dist-test-slave-4wm5"
server_key: "e72ad60478b6aedb1ecb0781de200dd5"
server_key_iv: "a72fe68fc158b6aeecd4cc79e740d2bb"
server_key_version: "encryptionkey@0"
I20260925 09:25:07.616066 23628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance:
uuid: "b60b56d702634dd88265b684f88c8283"
format_stamp: "Formatted at 2026-09-25 09:25:07 on dist-test-slave-4wm5"
server_key: "e72ad60478b6aedb1ecb0781de200dd5"
server_key_iv: "a72fe68fc158b6aeecd4cc79e740d2bb"
server_key_version: "encryptionkey@0"
I20260925 09:25:07.622653 23628 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260925 09:25:07.627024 23642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:07.628495 23628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:25:07.628681 23628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal
uuid: "b60b56d702634dd88265b684f88c8283"
format_stamp: "Formatted at 2026-09-25 09:25:07 on dist-test-slave-4wm5"
server_key: "e72ad60478b6aedb1ecb0781de200dd5"
server_key_iv: "a72fe68fc158b6aeecd4cc79e740d2bb"
server_key_version: "encryptionkey@0"
I20260925 09:25:07.628998 23628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:07.640781 23628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:07.641765 23628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:07.642143 23628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:07.643209 23628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:25:07.644814 23628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:25:07.644902 23628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:07.645058 23628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:25:07.645147 23628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:07.681154 23628 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.3:36065
I20260925 09:25:07.681210 23754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.3:36065 every 8 connection(s)
I20260925 09:25:07.682619 23628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/data/info.pb
I20260925 09:25:07.688457 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23628
I20260925 09:25:07.688598 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal/instance
I20260925 09:25:07.689095 3400 external_mini_cluster.cc:1428] Setting key cd00fc2e529c84f134e12dabf40a27ff
I20260925 09:25:07.696218 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.685439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.3:58573 (local address 127.3.82.62:38611)
0925 09:25:07.685630 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:07.685635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:07.687274 (+ 1639us) server_negotiation.cc:407] Connection header received
0925 09:25:07.689278 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:07.689284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:07.689375 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:07.689498 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:07.690942 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.691800 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.693516 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.693975 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.694073 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.695405 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:07.695409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:07.695412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:07.695430 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:07.695458 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:07.695469 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:07.695537 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:07.695891 (+ 354us) server_negotiation.cc:299] Negotiation successful
0925 09:25:07.695970 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:25:07.697372 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.686052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:38611 (local address 127.3.82.3:58573)
0925 09:25:07.687100 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:07.687137 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:07.688954 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:07.689777 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:07.689791 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:07.690055 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:07.690756 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.690782 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.691989 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.691995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.693377 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.693390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.694169 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.694174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.694374 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.695101 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 09:25:07.695104 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:07.695143 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 09:25:07.695229 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:07.695590 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:07.695601 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:07.695608 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 09:25:07.695860 (+ 252us) client_negotiation.cc:240] Negotiation successful
0925 09:25:07.696184 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":291,"threads_started":1}
I20260925 09:25:07.699189 23755 heartbeater.cc:347] Connected to a master server at 127.3.82.62:38611
I20260925 09:25:07.699563 23755 heartbeater.cc:464] Registering TS with master...
I20260925 09:25:07.700399 23755 heartbeater.cc:511] Master 127.3.82.62:38611 requested a full tablet report, sending...
I20260925 09:25:07.702157 23308 ts_manager.cc:194] Registered new tserver with Master: b60b56d702634dd88265b684f88c8283 (127.3.82.3:36065)
I20260925 09:25:07.703246 23308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.82.3:58573
I20260925 09:25:07.705080 3400 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:25:07.717659 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.710532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.3.82.62:38611)
0925 09:25:07.710948 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:07.710958 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:07.711350 (+ 392us) server_negotiation.cc:407] Connection header received
0925 09:25:07.711629 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:07.711634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:07.711803 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:07.711933 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:07.712553 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.713318 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.714932 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.715391 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.715544 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.716836 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:07.716841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:07.716845 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:07.716867 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:07.716902 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:07.716915 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:07.717021 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:07.717169 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 09:25:07.717283 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260925 09:25:07.722579 23308 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:25:07.729718 23308 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:25:07.732638 23494 heartbeater.cc:503] Master 127.3.82.62:38611 was elected leader, sending a full tablet report...
I20260925 09:25:07.757710 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.744232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.2:45687 (local address 127.0.0.1:40506)
0925 09:25:07.748301 (+ 4069us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:07.748328 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:07.748491 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:07.749472 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:07.749479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:07.749511 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:07.749958 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.749976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.752505 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.752509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.754400 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.754418 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.756707 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.756713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.756947 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.757005 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 09:25:07.757135 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 09:25:07.757349 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3897,"thread_start_us":3687,"threads_started":1}
I20260925 09:25:07.757874 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.744891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.3.82.2:45687)
0925 09:25:07.747732 (+ 2841us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:07.747746 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:07.748662 (+ 916us) server_negotiation.cc:407] Connection header received
0925 09:25:07.748811 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:07.748825 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:07.749196 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:07.749449 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:07.750254 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.752325 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.754632 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.756179 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.756307 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.756587 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:07.757246 (+ 659us) server_negotiation.cc:299] Negotiation successful
0925 09:25:07.757472 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2616,"thread_start_us":2449,"threads_started":1}
I20260925 09:25:07.758284 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.743823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:36065 (local address 127.0.0.1:57192)
0925 09:25:07.745871 (+ 2048us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:07.745946 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:07.746171 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:07.750859 (+ 4688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:07.750865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:07.750898 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:07.751284 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.751301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.753632 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.753639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.755641 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.755656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.757442 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.757448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.757636 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.757669 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 09:25:07.757736 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 09:25:07.757927 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1637,"mutex_wait_us":17,"thread_start_us":1462,"threads_started":1}
I20260925 09:25:07.759155 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.748535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:34705 (local address 127.0.0.1:41716)
0925 09:25:07.749038 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:07.749059 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:07.749177 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:07.751185 (+ 2008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:07.751190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:07.751214 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:07.751692 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.751708 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.753684 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.753692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.756017 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:07.756032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.758332 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.758340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:07.758595 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.758624 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 09:25:07.758720 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 09:25:07.758841 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":103,"thread_start_us":912,"threads_started":1}
I20260925 09:25:07.759346 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.749193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.3.82.1:34705)
0925 09:25:07.750578 (+ 1385us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:07.750586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:07.750610 (+ 24us) server_negotiation.cc:407] Connection header received
0925 09:25:07.750698 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:07.750706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:07.750908 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:07.751099 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:07.751899 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.753482 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.756273 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.757727 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.757908 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.758188 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:07.758794 (+ 606us) server_negotiation.cc:299] Negotiation successful
0925 09:25:07.758985 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1213,"thread_start_us":1052,"threads_started":1}
I20260925 09:25:07.759579 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:07.744184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.3.82.3:36065)
0925 09:25:07.750237 (+ 6053us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:07.750247 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:07.750288 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:25:07.750395 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:07.750406 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:07.750652 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:07.750855 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:07.751788 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.753483 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.755898 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:07.757224 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:07.758671 (+ 1447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:07.758958 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:07.759030 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 09:25:07.759201 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5872,"thread_start_us":5723,"threads_started":1}
I20260925 09:25:07.762590 23560 tablet_service.cc:1467] Processing CreateTablet for tablet 1853b6c32c1f49908ac080ed349724cf (DEFAULT_TABLE table=test-table [id=25cea9a7258c4019ae1efeaf2eafd208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:07.763183 23690 tablet_service.cc:1467] Processing CreateTablet for tablet 1853b6c32c1f49908ac080ed349724cf (DEFAULT_TABLE table=test-table [id=25cea9a7258c4019ae1efeaf2eafd208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:07.763183 23429 tablet_service.cc:1467] Processing CreateTablet for tablet 1853b6c32c1f49908ac080ed349724cf (DEFAULT_TABLE table=test-table [id=25cea9a7258c4019ae1efeaf2eafd208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:25:07.764024 23560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1853b6c32c1f49908ac080ed349724cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:07.764609 23690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1853b6c32c1f49908ac080ed349724cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:07.764611 23429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1853b6c32c1f49908ac080ed349724cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:07.777177 23773 tablet_bootstrap.cc:492] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Bootstrap starting.
I20260925 09:25:07.780561 23774 tablet_bootstrap.cc:492] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Bootstrap starting.
I20260925 09:25:07.781301 23773 tablet_bootstrap.cc:654] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:07.781652 23775 tablet_bootstrap.cc:492] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Bootstrap starting.
I20260925 09:25:07.782862 23773 log.cc:824] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:07.785717 23775 tablet_bootstrap.cc:654] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:07.786507 23773 tablet_bootstrap.cc:492] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: No bootstrap required, opened a new log
I20260925 09:25:07.786957 23773 ts_tablet_manager.cc:1397] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260925 09:25:07.787127 23775 log.cc:824] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:07.787581 23774 tablet_bootstrap.cc:654] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:07.789254 23774 log.cc:824] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:07.792768 23774 tablet_bootstrap.cc:492] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: No bootstrap required, opened a new log
I20260925 09:25:07.793112 23774 ts_tablet_manager.cc:1397] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260925 09:25:07.793165 23775 tablet_bootstrap.cc:492] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: No bootstrap required, opened a new log
I20260925 09:25:07.793437 23775 ts_tablet_manager.cc:1397] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260925 09:25:07.796890 23773 raft_consensus.cc:348] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:07.797308 23773 raft_consensus.cc:374] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:07.797451 23773 raft_consensus.cc:729] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906e7d262830442caa118357343f5aba, State: Initialized, Role: FOLLOWER
I20260925 09:25:07.798130 23773 consensus_queue.cc:260] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:07.799306 23625 heartbeater.cc:503] Master 127.3.82.62:38611 was elected leader, sending a full tablet report...
I20260925 09:25:07.800128 23773 ts_tablet_manager.cc:1428] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260925 09:25:07.803915 23774 raft_consensus.cc:348] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:07.804409 23774 raft_consensus.cc:374] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:07.804540 23774 raft_consensus.cc:729] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949f38a9aa954ff3bceb46bff0f9b755, State: Initialized, Role: FOLLOWER
I20260925 09:25:07.805306 23774 consensus_queue.cc:260] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:07.805343 23775 raft_consensus.cc:348] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:07.805785 23775 raft_consensus.cc:374] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:07.805923 23775 raft_consensus.cc:729] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60b56d702634dd88265b684f88c8283, State: Initialized, Role: FOLLOWER
I20260925 09:25:07.806566 23775 consensus_queue.cc:260] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:07.807372 23774 ts_tablet_manager.cc:1428] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260925 09:25:07.807717 23755 heartbeater.cc:503] Master 127.3.82.62:38611 was elected leader, sending a full tablet report...
I20260925 09:25:07.808871 23775 ts_tablet_manager.cc:1428] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260925 09:25:07.934392 23756 tablet.cc:2369] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:25:07.957434 23495 tablet.cc:2369] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:25:07.958101 23626 tablet.cc:2369] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:25:08.016086 23779 raft_consensus.cc:482] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:25:08.016412 23779 raft_consensus.cc:504] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:08.018391 23779 leader_election.cc:288] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60b56d702634dd88265b684f88c8283 (127.3.82.3:36065), 949f38a9aa954ff3bceb46bff0f9b755 (127.3.82.1:34705)
I20260925 09:25:08.025724 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.018795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:60785 (local address 127.3.82.3:36065)
0925 09:25:08.019027 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:08.019034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:08.019513 (+ 479us) server_negotiation.cc:407] Connection header received
0925 09:25:08.019622 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:08.019630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:08.019731 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:08.019856 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:08.020588 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.021552 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.023560 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.024625 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.024728 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.024807 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:08.025383 (+ 576us) server_negotiation.cc:299] Negotiation successful
0925 09:25:08.025496 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:25:08.025949 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.018614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.3:36065 (local address 127.3.82.2:60785)
0925 09:25:08.019303 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:08.019324 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:08.019476 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:08.020011 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:08.020016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:08.020045 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:08.020431 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:08.020445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.021689 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.021695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:08.023350 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:08.023363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.024907 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.024916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:08.025139 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.025182 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 09:25:08.025311 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 09:25:08.025486 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":383,"threads_started":1}
I20260925 09:25:08.026701 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.019872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.82.2:40921 (local address 127.3.82.1:34705)
0925 09:25:08.020046 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:08.020052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:08.020170 (+ 118us) server_negotiation.cc:407] Connection header received
0925 09:25:08.020306 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:08.020312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:08.020417 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:08.020523 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 09:25:08.021222 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.022171 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.024336 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.025620 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.025760 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.025942 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:08.026333 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 09:25:08.026472 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 09:25:08.026808 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.019459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.1:34705 (local address 127.3.82.2:40921)
0925 09:25:08.020063 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:08.020078 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:08.020196 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:08.020582 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:08.020586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:08.020612 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 09:25:08.021060 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:08.021074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.022347 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.022351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:08.024076 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:08.024091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.025959 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.025967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:08.026147 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.026183 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 09:25:08.026291 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 09:25:08.026467 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":5,"thread_start_us":275,"threads_started":1}
I20260925 09:25:08.027278 23710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1853b6c32c1f49908ac080ed349724cf" candidate_uuid: "906e7d262830442caa118357343f5aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60b56d702634dd88265b684f88c8283" is_pre_election: true
I20260925 09:25:08.027900 23710 raft_consensus.cc:2454] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906e7d262830442caa118357343f5aba in term 0.
I20260925 09:25:08.027895 23449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1853b6c32c1f49908ac080ed349724cf" candidate_uuid: "906e7d262830442caa118357343f5aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949f38a9aa954ff3bceb46bff0f9b755" is_pre_election: true
I20260925 09:25:08.028460 23449 raft_consensus.cc:2454] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906e7d262830442caa118357343f5aba in term 0.
I20260925 09:25:08.028707 23515 leader_election.cc:302] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906e7d262830442caa118357343f5aba, b60b56d702634dd88265b684f88c8283; no voters:
I20260925 09:25:08.029215 23779 raft_consensus.cc:2781] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:25:08.029414 23779 raft_consensus.cc:482] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:25:08.029546 23779 raft_consensus.cc:3037] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:08.031844 23779 raft_consensus.cc:504] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:08.032510 23779 leader_election.cc:288] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [CANDIDATE]: Term 1 election: Requested vote from peers b60b56d702634dd88265b684f88c8283 (127.3.82.3:36065), 949f38a9aa954ff3bceb46bff0f9b755 (127.3.82.1:34705)
I20260925 09:25:08.033253 23710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1853b6c32c1f49908ac080ed349724cf" candidate_uuid: "906e7d262830442caa118357343f5aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60b56d702634dd88265b684f88c8283"
I20260925 09:25:08.033504 23710 raft_consensus.cc:3037] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:08.033552 23449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1853b6c32c1f49908ac080ed349724cf" candidate_uuid: "906e7d262830442caa118357343f5aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949f38a9aa954ff3bceb46bff0f9b755"
I20260925 09:25:08.033807 23449 raft_consensus.cc:3037] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:08.036312 23710 raft_consensus.cc:2454] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906e7d262830442caa118357343f5aba in term 1.
I20260925 09:25:08.036599 23449 raft_consensus.cc:2454] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906e7d262830442caa118357343f5aba in term 1.
I20260925 09:25:08.036880 23515 leader_election.cc:302] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906e7d262830442caa118357343f5aba, b60b56d702634dd88265b684f88c8283; no voters:
I20260925 09:25:08.037307 23779 raft_consensus.cc:2781] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:08.037768 23779 raft_consensus.cc:686] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 1 LEADER]: Becoming Leader. State: Replica: 906e7d262830442caa118357343f5aba, State: Running, Role: LEADER
I20260925 09:25:08.038429 23779 consensus_queue.cc:237] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } }
I20260925 09:25:08.044251 23307 catalog_manager.cc:5495] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba reported cstate change: term changed from 0 to 1, leader changed from <none> to 906e7d262830442caa118357343f5aba (127.3.82.2). New cstate: current_term: 1 leader_uuid: "906e7d262830442caa118357343f5aba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906e7d262830442caa118357343f5aba" member_type: VOTER last_known_addr { host: "127.3.82.2" port: 45687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 } health_report { overall_health: UNKNOWN } } }
I20260925 09:25:08.128309 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.122190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40522 (local address 127.3.82.2:45687)
0925 09:25:08.122401 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:08.122405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:08.122423 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:25:08.122517 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:08.122520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:08.122610 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:08.122755 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:08.123139 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.123998 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.125633 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.126235 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.126402 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.126969 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:08.127571 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:08.127699 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:08.128020 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 09:25:08.128095 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 09:25:08.137967 23449 raft_consensus.cc:1261] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 1 FOLLOWER]: Refusing update from remote peer 906e7d262830442caa118357343f5aba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:08.138034 23710 raft_consensus.cc:1261] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 1 FOLLOWER]: Refusing update from remote peer 906e7d262830442caa118357343f5aba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:08.139189 23784 consensus_queue.cc:1035] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60b56d702634dd88265b684f88c8283" member_type: VOTER last_known_addr { host: "127.3.82.3" port: 36065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:08.139597 23779 consensus_queue.cc:1035] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [LEADER]: Connected to new peer: Peer: permanent_uuid: "949f38a9aa954ff3bceb46bff0f9b755" member_type: VOTER last_known_addr { host: "127.3.82.1" port: 34705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:08.174240 23308 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:25:08.174577 23308 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 09:25:08.179214 23308 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 215143838aaa49828c5eed4284fb3cac: Sending DeleteTablet for 3 replicas of tablet 1853b6c32c1f49908ac080ed349724cf
I20260925 09:25:08.180636 23560 tablet_service.cc:1514] Processing DeleteTablet for tablet 1853b6c32c1f49908ac080ed349724cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:08 UTC) from {username='slave'} at 127.0.0.1:40506
I20260925 09:25:08.180819 23429 tablet_service.cc:1514] Processing DeleteTablet for tablet 1853b6c32c1f49908ac080ed349724cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:08 UTC) from {username='slave'} at 127.0.0.1:41716
I20260925 09:25:08.181025 23690 tablet_service.cc:1514] Processing DeleteTablet for tablet 1853b6c32c1f49908ac080ed349724cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:25:08 UTC) from {username='slave'} at 127.0.0.1:57192
I20260925 09:25:08.181991 23798 tablet_replica.cc:330] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: stopping tablet replica
I20260925 09:25:08.182658 23798 raft_consensus.cc:2229] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:25:08.183421 23798 raft_consensus.cc:2258] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:08.184374 23797 tablet_replica.cc:330] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: stopping tablet replica
I20260925 09:25:08.184777 23797 raft_consensus.cc:2229] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:25:08.185345 23797 raft_consensus.cc:2258] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:08.185684 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23367
I20260925 09:25:08.188239 23799 tablet_replica.cc:330] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: stopping tablet replica
I20260925 09:25:08.188741 23799 raft_consensus.cc:2229] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:25:08.189486 23799 raft_consensus.cc:2258] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:25:08.189643 23797 ts_tablet_manager.cc:1905] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:08.190210 23798 ts_tablet_manager.cc:1905] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:08.194406 23799 ts_tablet_manager.cc:1905] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:25:08.197448 23797 ts_tablet_manager.cc:1918] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:08.197733 23797 log.cc:1196] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-1/wal/wals/1853b6c32c1f49908ac080ed349724cf
I20260925 09:25:08.198427 23797 ts_tablet_manager.cc:1939] T 1853b6c32c1f49908ac080ed349724cf P 906e7d262830442caa118357343f5aba: Deleting consensus metadata
I20260925 09:25:08.198449 23798 ts_tablet_manager.cc:1918] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:08.198719 23798 log.cc:1196] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-0/wal/wals/1853b6c32c1f49908ac080ed349724cf
I20260925 09:25:08.199321 23798 ts_tablet_manager.cc:1939] T 1853b6c32c1f49908ac080ed349724cf P 949f38a9aa954ff3bceb46bff0f9b755: Deleting consensus metadata
I20260925 09:25:08.200510 23293 catalog_manager.cc:4841] TS 906e7d262830442caa118357343f5aba (127.3.82.2:45687): tablet 1853b6c32c1f49908ac080ed349724cf (table test-table [id=25cea9a7258c4019ae1efeaf2eafd208]) successfully deleted
I20260925 09:25:08.201658 23294 catalog_manager.cc:4841] TS 949f38a9aa954ff3bceb46bff0f9b755 (127.3.82.1:34705): tablet 1853b6c32c1f49908ac080ed349724cf (table test-table [id=25cea9a7258c4019ae1efeaf2eafd208]) successfully deleted
I20260925 09:25:08.206252 23799 ts_tablet_manager.cc:1918] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:25:08.206494 23799 log.cc:1196] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Deleting WAL directory at /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790327973894689-3400-0/minicluster-data/ts-2/wal/wals/1853b6c32c1f49908ac080ed349724cf
I20260925 09:25:08.207085 23799 ts_tablet_manager.cc:1939] T 1853b6c32c1f49908ac080ed349724cf P b60b56d702634dd88265b684f88c8283: Deleting consensus metadata
I20260925 09:25:08.209159 23294 catalog_manager.cc:4841] TS b60b56d702634dd88265b684f88c8283 (127.3.82.3:36065): tablet 1853b6c32c1f49908ac080ed349724cf (table test-table [id=25cea9a7258c4019ae1efeaf2eafd208]) successfully deleted
I20260925 09:25:08.210729 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.187169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41718 (local address 127.3.82.1:34705)
0925 09:25:08.187360 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:08.187366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:08.202071 (+ 14705us) server_negotiation.cc:407] Connection header received
0925 09:25:08.202260 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:08.202267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:08.202392 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:08.202508 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:08.203116 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.204279 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.206351 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.207100 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.207212 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.208940 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:08.208950 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:08.208968 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:08.209815 (+ 847us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:08.209937 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:08.209958 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:08.210081 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:08.210238 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 09:25:08.210359 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:25:08.211374 23489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:08.304237 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23367
I20260925 09:25:08.323390 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23498
I20260925 09:25:08.332876 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.324283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.3.82.2:45687)
0925 09:25:08.324489 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:08.324496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:08.324805 (+ 309us) server_negotiation.cc:407] Connection header received
0925 09:25:08.325018 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:08.325024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:08.325120 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:08.325241 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:08.325648 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.326842 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.328936 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.329472 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.329596 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.330979 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:08.330989 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:08.331005 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:08.331807 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:08.331866 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:08.331890 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:08.331989 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:08.332338 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 09:25:08.332499 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:25:08.333746 23620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:08.450768 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23498
I20260925 09:25:08.472946 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23628
I20260925 09:25:08.481709 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:08.474018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.3.82.3:36065)
0925 09:25:08.474254 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:08.474261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:08.474283 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:25:08.474363 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:08.474367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:08.474452 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:08.474582 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:08.475028 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.475843 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.477511 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:08.478437 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:08.478611 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:08.479977 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:08.479989 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:08.480011 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:08.480720 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:08.480774 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:08.480795 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:08.480933 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:08.481162 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 09:25:08.481307 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260925 09:25:08.482436 23750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:08.576839 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23628
I20260925 09:25:08.595423 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23277
I20260925 09:25:08.596683 23338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:08.710487 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23277
2026-09-25T09:25:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3166 ms)
[----------] 6 tests from EncryptionPolicyTest (23102 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:08 dist-test-slave-4wm5 krb5kdc[23807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:08 dist-test-slave-4wm5 krb5kdc[23807](info): set up 2 sockets
Sep 25 09:25:08 dist-test-slave-4wm5 krb5kdc[23807](info): commencing operation
krb5kdc: starting...
W20260925 09:25:09.842581 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:10 dist-test-slave-4wm5 krb5kdc[23807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:10.130486 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:10.489415 23820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:10.489888 23820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:10.490029 23820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:10.497581 23820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:10.497704 23820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:10.497751 23820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:10.497804 23820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:10.497937 23820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:10.508608 23820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:10.509967 23820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:10.511278 23820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:10.524523 23825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:10.524489 23826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:10.524463 23828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:10.525110 23820 server_base.cc:1034] running on GCE node
I20260925 09:25:10.526091 23820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:10.527566 23820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:10.528829 23820 hybrid_clock.cc:648] HybridClock initialized: now 1790328310528786 us; error 36 us; skew 500 ppm
I20260925 09:25:10.529349 23820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:10 dist-test-slave-4wm5 krb5kdc[23807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328310, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:10.543321 23820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:10.545168 23820 webserver.cc:460] Webserver started at http://127.0.0.1:33391/ using document root <none> and password file <none>
I20260925 09:25:10.546028 23820 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:10.546167 23820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:10.546514 23820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:10.549777 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "c697a91cb2bc49d6b1d5cb28665dc022"
format_stamp: "Formatted at 2026-09-25 09:25:10 on dist-test-slave-4wm5"
server_key: "3a618f20a74534a878a225b23d61cd79"
server_key_iv: "b1a69eed524159107f85b826754331a5"
server_key_version: "encryptionkey@0"
I20260925 09:25:10.550769 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "c697a91cb2bc49d6b1d5cb28665dc022"
format_stamp: "Formatted at 2026-09-25 09:25:10 on dist-test-slave-4wm5"
server_key: "3a618f20a74534a878a225b23d61cd79"
server_key_iv: "b1a69eed524159107f85b826754331a5"
server_key_version: "encryptionkey@0"
I20260925 09:25:10.556814 23820 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260925 09:25:10.561232 23835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:10.563057 23820 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:10.563315 23820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "c697a91cb2bc49d6b1d5cb28665dc022"
format_stamp: "Formatted at 2026-09-25 09:25:10 on dist-test-slave-4wm5"
server_key: "3a618f20a74534a878a225b23d61cd79"
server_key_iv: "b1a69eed524159107f85b826754331a5"
server_key_version: "encryptionkey@0"
I20260925 09:25:10.563583 23820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:10.582031 23820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:10.587472 23820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:10.587829 23820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:10.608831 23820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40829
I20260925 09:25:10.608839 23886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40829 every 8 connection(s)
I20260925 09:25:10.610358 23820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:10.614255 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:10.619978 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23820
I20260925 09:25:10.624457 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:10.625172 3400 external_mini_cluster.cc:1428] Setting key 104ba50a8d6f1e8252880f98174be753
I20260925 09:25:10.625452 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022: Bootstrap starting.
I20260925 09:25:10.629408 23887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:10.630687 23887 log.cc:824] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:10.634153 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022: No bootstrap required, opened a new log
Sep 25 09:25:10 dist-test-slave-4wm5 krb5kdc[23807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:25:10.644316 23887 raft_consensus.cc:348] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260925 09:25:10.644942 23887 raft_consensus.cc:374] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:10.645087 23887 raft_consensus.cc:729] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c697a91cb2bc49d6b1d5cb28665dc022, State: Initialized, Role: FOLLOWER
I20260925 09:25:10.645792 23887 consensus_queue.cc:260] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260925 09:25:10.646135 23887 raft_consensus.cc:388] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:10.646328 23887 raft_consensus.cc:482] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:10.646540 23887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:10.648916 23887 raft_consensus.cc:504] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260925 09:25:10.649506 23887 leader_election.cc:302] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c697a91cb2bc49d6b1d5cb28665dc022; no voters:
I20260925 09:25:10.650643 23887 leader_election.cc:288] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:10.650816 23892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:10.651551 23892 raft_consensus.cc:686] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [term 1 LEADER]: Becoming Leader. State: Replica: c697a91cb2bc49d6b1d5cb28665dc022, State: Running, Role: LEADER
I20260925 09:25:10.651810 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:10.627316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57381 (local address 127.0.0.1:40829)
0925 09:25:10.628218 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:10.628230 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:10.628272 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:25:10.630328 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:10.630352 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:10.630820 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:10.631285 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:10.632958 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:10.634469 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:10.636444 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:10.637208 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:10.637369 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:10.642458 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:10.642468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:10.642487 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:10.642537 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:10.646618 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:10.647631 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:10.647644 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:10.647661 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:10.647865 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:10.648381 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:10.648386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:10.648390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:10.648995 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:10.649329 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:10.649923 (+ 594us) server_negotiation.cc:299] Negotiation successful
0925 09:25:10.650280 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":436,"threads_started":1}
I20260925 09:25:10.652585 23892 consensus_queue.cc:237] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } }
I20260925 09:25:10.654484 23887 sys_catalog.cc:564] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:10.655609 23894 sys_catalog.cc:455] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } } }
I20260925 09:25:10.656035 23894 sys_catalog.cc:458] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:10.656728 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:10.656697 23893 sys_catalog.cc:455] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c697a91cb2bc49d6b1d5cb28665dc022. Latest consensus state: current_term: 1 leader_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c697a91cb2bc49d6b1d5cb28665dc022" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40829 } } }
I20260925 09:25:10.657050 23893 sys_catalog.cc:458] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:10.667452 23897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:10.681478 23897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:10.684855 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:10.664338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46913 (local address 127.0.0.1:40829)
0925 09:25:10.664538 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:10.664547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:10.666458 (+ 1911us) server_negotiation.cc:407] Connection header received
0925 09:25:10.666717 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:10.666723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:10.666814 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:10.666892 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:10.668786 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:10.669662 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:10.671952 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:10.672446 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:10.672605 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:10.676069 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:10.676074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:10.676077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:10.676118 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:10.680456 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:10.681908 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:10.681914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:10.681918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:10.682099 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:10.682734 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:10.682743 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:10.682746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:10.683217 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:10.683502 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:10.684246 (+ 744us) server_negotiation.cc:299] Negotiation successful
0925 09:25:10.684461 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:25:10.700968 23897 catalog_manager.cc:1347] Generated new cluster ID: a77b46251857460b9b1af5de76891830
I20260925 09:25:10.701280 23897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:10.717536 23897 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:10.719506 23897 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:10.727113 23897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c697a91cb2bc49d6b1d5cb28665dc022: Generated new TSK 0
I20260925 09:25:10.728360 23897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:10.739517 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23820
I20260925 09:25:10.741088 23882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:10.837082 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23820
W20260925 09:25:10.855450 23805 connection.cc:535] client connection to 127.0.0.1:40829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40829): Connection reset by peer (error 104)
2026-09-25T09:25:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2121 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:10 dist-test-slave-4wm5 krb5kdc[23920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:10 dist-test-slave-4wm5 krb5kdc[23920](info): set up 2 sockets
Sep 25 09:25:10 dist-test-slave-4wm5 krb5kdc[23920](info): commencing operation
krb5kdc: starting...
W20260925 09:25:11.966437 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:12 dist-test-slave-4wm5 krb5kdc[23920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:12.249878 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:12.603545 23933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:12.603878 23933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:12.604049 23933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:12.611619 23933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:12.611758 23933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:12.611855 23933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:12.611940 23933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:12.612005 23933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:12.623571 23933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:12.625509 23933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:12.627615 23933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:12.642230 23939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:12.642170 23941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:12.642287 23938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:12.643345 23933 server_base.cc:1034] running on GCE node
I20260925 09:25:12.644305 23933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:12.646003 23933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:12.647305 23933 hybrid_clock.cc:648] HybridClock initialized: now 1790328312647258 us; error 41 us; skew 500 ppm
I20260925 09:25:12.647814 23933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:12 dist-test-slave-4wm5 krb5kdc[23920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:12.651444 23933 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:12.653334 23933 webserver.cc:460] Webserver started at http://127.0.0.1:38399/ using document root <none> and password file <none>
I20260925 09:25:12.654210 23933 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:12.654361 23933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:12.654680 23933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:12.658490 23933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "a480ec60fab04a50be3647249b37327f"
format_stamp: "Formatted at 2026-09-25 09:25:12 on dist-test-slave-4wm5"
server_key: "680b994a745d484b27e2661e363210a6"
server_key_iv: "ceb3ae07536d28f06f0caf465f9edabf"
server_key_version: "encryptionkey@0"
I20260925 09:25:12.659425 23933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "a480ec60fab04a50be3647249b37327f"
format_stamp: "Formatted at 2026-09-25 09:25:12 on dist-test-slave-4wm5"
server_key: "680b994a745d484b27e2661e363210a6"
server_key_iv: "ceb3ae07536d28f06f0caf465f9edabf"
server_key_version: "encryptionkey@0"
I20260925 09:25:12.666013 23933 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260925 09:25:12.670217 23948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:12.671809 23933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 09:25:12.672027 23933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "a480ec60fab04a50be3647249b37327f"
format_stamp: "Formatted at 2026-09-25 09:25:12 on dist-test-slave-4wm5"
server_key: "680b994a745d484b27e2661e363210a6"
server_key_iv: "ceb3ae07536d28f06f0caf465f9edabf"
server_key_version: "encryptionkey@0"
I20260925 09:25:12.672289 23933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:12.695308 23933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:12.699959 23933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:12.700282 23933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:12.720078 23933 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40861
I20260925 09:25:12.720110 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40861 every 8 connection(s)
I20260925 09:25:12.721557 23933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:12.725438 24000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:12.727372 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 23933
I20260925 09:25:12.727705 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:12.728272 3400 external_mini_cluster.cc:1428] Setting key 4221b3605e7762610dc84c341c183a8c
I20260925 09:25:12.740244 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f: Bootstrap starting.
I20260925 09:25:12.742610 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:12.730250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34433 (local address 127.0.0.1:40861)
0925 09:25:12.731144 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:12.731157 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:12.731198 (+ 41us) server_negotiation.cc:407] Connection header received
0925 09:25:12.732855 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:12.732884 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:12.733298 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:12.733636 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:12.734256 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:12.735574 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:12.737574 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:12.738401 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:12.738540 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:12.739953 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:12.739960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:12.739975 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:12.740656 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:12.740737 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:12.740770 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:12.740902 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:12.741249 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 09:25:12.741487 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":392,"threads_started":1}
I20260925 09:25:12.743803 24000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:12.744997 24000 log.cc:824] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:12.747104 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:12.749104 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f: No bootstrap required, opened a new log
I20260925 09:25:12.760042 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:12.752607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38781 (local address 127.0.0.1:40861)
0925 09:25:12.752885 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:12.752890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:12.753131 (+ 241us) server_negotiation.cc:407] Connection header received
0925 09:25:12.753329 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:12.753333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:12.753407 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:12.753495 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:12.754180 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:12.754994 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:12.757029 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:12.757499 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:12.757615 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:12.759052 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:12.759057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:12.759061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:12.759081 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:12.759149 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:12.759167 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:12.759256 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:12.759512 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 09:25:12.759610 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 09:25:12.760288 24000 raft_consensus.cc:348] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a480ec60fab04a50be3647249b37327f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260925 09:25:12.760752 24000 raft_consensus.cc:374] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:12.760878 24000 raft_consensus.cc:729] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a480ec60fab04a50be3647249b37327f, State: Initialized, Role: FOLLOWER
I20260925 09:25:12.761637 24000 consensus_queue.cc:260] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a480ec60fab04a50be3647249b37327f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260925 09:25:12.761932 24000 raft_consensus.cc:388] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:12.762087 24000 raft_consensus.cc:482] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:12.762269 24000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:12.764581 24000 raft_consensus.cc:504] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a480ec60fab04a50be3647249b37327f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260925 09:25:12.765148 24000 leader_election.cc:302] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a480ec60fab04a50be3647249b37327f; no voters:
I20260925 09:25:12.765930 24000 leader_election.cc:288] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:12.766113 24010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:12.767347 24010 raft_consensus.cc:686] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [term 1 LEADER]: Becoming Leader. State: Replica: a480ec60fab04a50be3647249b37327f, State: Running, Role: LEADER
I20260925 09:25:12.768126 24010 consensus_queue.cc:237] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a480ec60fab04a50be3647249b37327f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260925 09:25:12.768565 24000 sys_catalog.cc:564] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:12.771678 24011 sys_catalog.cc:455] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a480ec60fab04a50be3647249b37327f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a480ec60fab04a50be3647249b37327f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } } }
I20260925 09:25:12.772352 24011 sys_catalog.cc:458] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:12.772953 24012 sys_catalog.cc:455] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a480ec60fab04a50be3647249b37327f. Latest consensus state: current_term: 1 leader_uuid: "a480ec60fab04a50be3647249b37327f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a480ec60fab04a50be3647249b37327f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } } }
I20260925 09:25:12.773314 24012 sys_catalog.cc:458] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:12.778472 24018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:12.784482 24018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:12.794087 24018 catalog_manager.cc:1347] Generated new cluster ID: d43bb24bdbc940f59226d8bb6f872779
I20260925 09:25:12.794288 24018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:12.829582 24018 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:12.831106 24018 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:12.841777 24018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a480ec60fab04a50be3647249b37327f: Generated new TSK 0
I20260925 09:25:12.842876 24018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:12.851329 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23933
W20260925 09:25:12.852541 23995 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34433
W20260925 09:25:12.853353 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 23933: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:12.853586 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 23933
W20260925 09:25:12.871618 23918 connection.cc:535] client connection to 127.0.0.1:40861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40861): Connection reset by peer (error 104)
2026-09-25T09:25:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:12 dist-test-slave-4wm5 krb5kdc[24031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:12 dist-test-slave-4wm5 krb5kdc[24031](info): set up 2 sockets
Sep 25 09:25:12 dist-test-slave-4wm5 krb5kdc[24031](info): commencing operation
krb5kdc: starting...
W20260925 09:25:14.014765 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:14 dist-test-slave-4wm5 krb5kdc[24031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:14.297286 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:14.647228 24044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:14.647578 24044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:14.647676 24044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:14.655112 24044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:14.655243 24044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:14.655290 24044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:14.655354 24044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:14.655391 24044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:14.665647 24044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:14.667102 24044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:14.668470 24044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:14.680989 24050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:14.680891 24049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:14.681041 24052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:14.682415 24044 server_base.cc:1034] running on GCE node
I20260925 09:25:14.683321 24044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:14.684943 24044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:14.686188 24044 hybrid_clock.cc:648] HybridClock initialized: now 1790328314686136 us; error 49 us; skew 500 ppm
I20260925 09:25:14.686575 24044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:14 dist-test-slave-4wm5 krb5kdc[24031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:14.690336 24044 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:14.692281 24044 webserver.cc:460] Webserver started at http://127.0.0.1:44985/ using document root <none> and password file <none>
I20260925 09:25:14.693019 24044 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:14.693127 24044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:14.693382 24044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:14.696529 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "206c663ba49a4ad3b6bb204e2968eb57"
format_stamp: "Formatted at 2026-09-25 09:25:14 on dist-test-slave-4wm5"
server_key: "8b83eff7bd5bd540d48e65a9fd3bf727"
server_key_iv: "4ac63cdca3ac9e10e534851a19457c44"
server_key_version: "encryptionkey@0"
I20260925 09:25:14.697283 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "206c663ba49a4ad3b6bb204e2968eb57"
format_stamp: "Formatted at 2026-09-25 09:25:14 on dist-test-slave-4wm5"
server_key: "8b83eff7bd5bd540d48e65a9fd3bf727"
server_key_iv: "4ac63cdca3ac9e10e534851a19457c44"
server_key_version: "encryptionkey@0"
I20260925 09:25:14.703081 24044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 09:25:14.707230 24059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:14.708947 24044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 09:25:14.709121 24044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "206c663ba49a4ad3b6bb204e2968eb57"
format_stamp: "Formatted at 2026-09-25 09:25:14 on dist-test-slave-4wm5"
server_key: "8b83eff7bd5bd540d48e65a9fd3bf727"
server_key_iv: "4ac63cdca3ac9e10e534851a19457c44"
server_key_version: "encryptionkey@0"
I20260925 09:25:14.709340 24044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:14.715827 24044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:14.727226 24044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:14.727551 24044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:14.748852 24044 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35261
I20260925 09:25:14.748872 24110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35261 every 8 connection(s)
I20260925 09:25:14.750511 24044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:14.754750 24111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:14.755803 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24044
I20260925 09:25:14.755990 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:14.756687 3400 external_mini_cluster.cc:1428] Setting key a1a9c5dd9771ff6afea44f83d711dd0d
I20260925 09:25:14.765470 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:14.758691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52145 (local address 127.0.0.1:35261)
0925 09:25:14.759694 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:14.759706 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:14.759754 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:25:14.761729 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:14.761756 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:14.762244 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:14.762627 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:14.763840 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:14.763847 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:14.763862 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:14.763878 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:14.763946 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:14.763971 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:14.764143 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:14.764266 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 09:25:14.764491 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":412,"threads_started":1}
I20260925 09:25:14.770005 24111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57: Bootstrap starting.
I20260925 09:25:14.770236 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:14.774324 24111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:14.775847 24111 log.cc:824] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:14.778947 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:14.775893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43527 (local address 127.0.0.1:35261)
0925 09:25:14.776105 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:14.776112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:14.776596 (+ 484us) server_negotiation.cc:407] Connection header received
0925 09:25:14.776807 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:14.776811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:14.776869 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:14.776973 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:14.778112 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:14.778120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:14.778127 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:14.778149 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:14.778196 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:14.778215 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:14.778345 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:14.778492 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 09:25:14.778593 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 09:25:14.779711 24111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57: No bootstrap required, opened a new log
I20260925 09:25:14.787737 24111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c663ba49a4ad3b6bb204e2968eb57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35261 } }
I20260925 09:25:14.788159 24111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:14.788251 24111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 206c663ba49a4ad3b6bb204e2968eb57, State: Initialized, Role: FOLLOWER
I20260925 09:25:14.788818 24111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c663ba49a4ad3b6bb204e2968eb57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35261 } }
I20260925 09:25:14.789044 24111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:14.789145 24111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:14.789273 24111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:14.791610 24111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c663ba49a4ad3b6bb204e2968eb57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35261 } }
I20260925 09:25:14.792416 24111 leader_election.cc:302] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 206c663ba49a4ad3b6bb204e2968eb57; no voters:
I20260925 09:25:14.793301 24111 leader_election.cc:288] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:14.793417 24121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:14.794713 24121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [term 1 LEADER]: Becoming Leader. State: Replica: 206c663ba49a4ad3b6bb204e2968eb57, State: Running, Role: LEADER
I20260925 09:25:14.795363 24121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c663ba49a4ad3b6bb204e2968eb57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35261 } }
I20260925 09:25:14.796008 24111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:14.799223 24122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "206c663ba49a4ad3b6bb204e2968eb57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c663ba49a4ad3b6bb204e2968eb57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35261 } } }
I20260925 09:25:14.799755 24122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:14.799680 24123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 206c663ba49a4ad3b6bb204e2968eb57. Latest consensus state: current_term: 1 leader_uuid: "206c663ba49a4ad3b6bb204e2968eb57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c663ba49a4ad3b6bb204e2968eb57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35261 } } }
I20260925 09:25:14.799959 24123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:14.800781 24128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:14.807605 24128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:14.816576 24128 catalog_manager.cc:1347] Generated new cluster ID: 8922e0cbdb984c5185bad4de9e255826
I20260925 09:25:14.816713 24128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:14.850409 24128 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:14.850979 24128 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:14.862619 24128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 206c663ba49a4ad3b6bb204e2968eb57: Generated new TSK 0
I20260925 09:25:14.863847 24128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:14.879974 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24044
W20260925 09:25:14.881165 24106 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52145
W20260925 09:25:14.882711 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24044: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:14.883179 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24044
2026-09-25T09:25:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:15 dist-test-slave-4wm5 krb5kdc[24142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:15 dist-test-slave-4wm5 krb5kdc[24142](info): set up 2 sockets
Sep 25 09:25:15 dist-test-slave-4wm5 krb5kdc[24142](info): commencing operation
krb5kdc: starting...
W20260925 09:25:16.086849 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:16 dist-test-slave-4wm5 krb5kdc[24142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:16.375308 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:16.729053 24155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:16.729444 24155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:16.729628 24155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:16.737227 24155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:16.737433 24155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:16.737530 24155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:16.737620 24155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:16.737720 24155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:16.748319 24155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:16.749778 24155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:16.751238 24155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:16.764627 24160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:16.764703 24163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:16.764721 24161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:16.765950 24155 server_base.cc:1034] running on GCE node
I20260925 09:25:16.766875 24155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:16.768544 24155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:16.769979 24155 hybrid_clock.cc:648] HybridClock initialized: now 1790328316769819 us; error 71 us; skew 500 ppm
I20260925 09:25:16.770493 24155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:16 dist-test-slave-4wm5 krb5kdc[24142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328316, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:16.790264 24155 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:16.792346 24155 webserver.cc:460] Webserver started at http://127.0.0.1:40807/ using document root <none> and password file <none>
I20260925 09:25:16.793222 24155 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:16.793424 24155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:16.793783 24155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:16.797283 24155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "83f5b1e75bb241199937e30cce827e4b"
format_stamp: "Formatted at 2026-09-25 09:25:16 on dist-test-slave-4wm5"
server_key: "53820841add0d47dfb0618191949378f"
server_key_iv: "d208a2f5b342287c151cd5023394f742"
server_key_version: "encryptionkey@0"
I20260925 09:25:16.798216 24155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "83f5b1e75bb241199937e30cce827e4b"
format_stamp: "Formatted at 2026-09-25 09:25:16 on dist-test-slave-4wm5"
server_key: "53820841add0d47dfb0618191949378f"
server_key_iv: "d208a2f5b342287c151cd5023394f742"
server_key_version: "encryptionkey@0"
I20260925 09:25:16.804370 24155 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260925 09:25:16.808878 24170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:16.810623 24155 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:16.810861 24155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "83f5b1e75bb241199937e30cce827e4b"
format_stamp: "Formatted at 2026-09-25 09:25:16 on dist-test-slave-4wm5"
server_key: "53820841add0d47dfb0618191949378f"
server_key_iv: "d208a2f5b342287c151cd5023394f742"
server_key_version: "encryptionkey@0"
I20260925 09:25:16.811118 24155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:16.840152 24155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:16.845624 24155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:16.846068 24155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:16.866494 24155 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35995
I20260925 09:25:16.866544 24221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35995 every 8 connection(s)
I20260925 09:25:16.868047 24155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:16.872579 24222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:16.874828 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24155
I20260925 09:25:16.875046 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:16.875597 3400 external_mini_cluster.cc:1428] Setting key 79a8226b87fafe57d12c323333631da5
I20260925 09:25:16.886693 24222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b: Bootstrap starting.
I20260925 09:25:16.891214 24222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b: Neither blocks nor log segments found. Creating new log.
Sep 25 09:25:16 dist-test-slave-4wm5 krb5kdc[24142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:25:16.892650 24222 log.cc:824] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:16.897089 24222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b: No bootstrap required, opened a new log
I20260925 09:25:16.905073 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:16.877675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40557 (local address 127.0.0.1:35995)
0925 09:25:16.878695 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:16.878713 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:16.878770 (+ 57us) server_negotiation.cc:407] Connection header received
0925 09:25:16.880618 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:16.880645 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:16.881133 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:16.881520 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:16.883657 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:16.885387 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:16.887463 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:16.888376 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:16.888570 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:16.894538 (+ 5968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:16.894550 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:16.894576 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:16.894630 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:16.899273 (+ 4643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:16.900282 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:16.900292 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:16.900303 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:16.900561 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:16.901075 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:16.901081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:16.901085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:16.902016 (+ 931us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:16.902352 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:16.902893 (+ 541us) server_negotiation.cc:299] Negotiation successful
0925 09:25:16.903476 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":439,"threads_started":1}
I20260925 09:25:16.908471 24222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f5b1e75bb241199937e30cce827e4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260925 09:25:16.908998 24222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:16.909130 24222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f5b1e75bb241199937e30cce827e4b, State: Initialized, Role: FOLLOWER
I20260925 09:25:16.909924 24222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f5b1e75bb241199937e30cce827e4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260925 09:25:16.910246 24222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:16.910413 24222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:16.910594 24222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:16.910646 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:16.912957 24222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f5b1e75bb241199937e30cce827e4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260925 09:25:16.913636 24222 leader_election.cc:302] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83f5b1e75bb241199937e30cce827e4b; no voters:
I20260925 09:25:16.914427 24222 leader_election.cc:288] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:16.914551 24230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:16.917723 24222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:16.918548 24230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [term 1 LEADER]: Becoming Leader. State: Replica: 83f5b1e75bb241199937e30cce827e4b, State: Running, Role: LEADER
I20260925 09:25:16.919381 24230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f5b1e75bb241199937e30cce827e4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260925 09:25:16.923714 24233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83f5b1e75bb241199937e30cce827e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f5b1e75bb241199937e30cce827e4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } } }
I20260925 09:25:16.924237 24233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:16.925602 24232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83f5b1e75bb241199937e30cce827e4b. Latest consensus state: current_term: 1 leader_uuid: "83f5b1e75bb241199937e30cce827e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f5b1e75bb241199937e30cce827e4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } } }
I20260925 09:25:16.927487 24232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:16.940490 24240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:16.943827 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:16.916076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33267 (local address 127.0.0.1:35995)
0925 09:25:16.916374 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:16.916384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:16.918523 (+ 2139us) server_negotiation.cc:407] Connection header received
0925 09:25:16.918838 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:16.918844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:16.918928 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:16.918994 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:16.921143 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:16.922063 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:16.924178 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:16.925955 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:16.926178 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:16.933528 (+ 7350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:16.933542 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:16.933551 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:16.933627 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:16.937769 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:16.939060 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:16.939078 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:16.939081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:16.939385 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:16.941800 (+ 2415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:16.941806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:16.941809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:16.942363 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:16.942648 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:16.943172 (+ 524us) server_negotiation.cc:299] Negotiation successful
0925 09:25:16.943436 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 09:25:16.946269 24240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:16.955144 24240 catalog_manager.cc:1347] Generated new cluster ID: bdb848e37272479e9c29c855b596f106
I20260925 09:25:16.955288 24240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:16.969264 24240 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:16.971315 24240 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:16.980793 24240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83f5b1e75bb241199937e30cce827e4b: Generated new TSK 0
I20260925 09:25:16.982208 24240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:16.993465 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24155
I20260925 09:25:16.995330 24217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:17.092971 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24155
2026-09-25T09:25:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2202 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:17 dist-test-slave-4wm5 krb5kdc[24255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:17 dist-test-slave-4wm5 krb5kdc[24255](info): set up 2 sockets
Sep 25 09:25:17 dist-test-slave-4wm5 krb5kdc[24255](info): commencing operation
krb5kdc: starting...
W20260925 09:25:18.226920 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:18 dist-test-slave-4wm5 krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:18.514124 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:18.869191 24268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:18.869511 24268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:18.869608 24268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:18.878139 24268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:18.878296 24268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:18.878350 24268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:18.878413 24268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:18.878464 24268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:18.889755 24268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:18.891204 24268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:18.892545 24268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:18.905508 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:18.905581 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:18.905539 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:18.906388 24268 server_base.cc:1034] running on GCE node
I20260925 09:25:18.907399 24268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:18.909231 24268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:18.910523 24268 hybrid_clock.cc:648] HybridClock initialized: now 1790328318910451 us; error 72 us; skew 500 ppm
I20260925 09:25:18.910948 24268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:18 dist-test-slave-4wm5 krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328318, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:18.914749 24268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:18.916522 24268 webserver.cc:460] Webserver started at http://127.0.0.1:33323/ using document root <none> and password file <none>
I20260925 09:25:18.917462 24268 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:18.917652 24268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:18.918047 24268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:18.921226 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "826ff26eee234c809931ba29aa3d745d"
format_stamp: "Formatted at 2026-09-25 09:25:18 on dist-test-slave-4wm5"
server_key: "7b2b7a8978201ac73ca5a59e64d7fde6"
server_key_iv: "cd141bbe808b088166864b29c009f2fc"
server_key_version: "encryptionkey@0"
I20260925 09:25:18.922120 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "826ff26eee234c809931ba29aa3d745d"
format_stamp: "Formatted at 2026-09-25 09:25:18 on dist-test-slave-4wm5"
server_key: "7b2b7a8978201ac73ca5a59e64d7fde6"
server_key_iv: "cd141bbe808b088166864b29c009f2fc"
server_key_version: "encryptionkey@0"
I20260925 09:25:18.928059 24268 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260925 09:25:18.932538 24283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:18.934193 24268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 09:25:18.934451 24268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "826ff26eee234c809931ba29aa3d745d"
format_stamp: "Formatted at 2026-09-25 09:25:18 on dist-test-slave-4wm5"
server_key: "7b2b7a8978201ac73ca5a59e64d7fde6"
server_key_iv: "cd141bbe808b088166864b29c009f2fc"
server_key_version: "encryptionkey@0"
I20260925 09:25:18.934705 24268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:18.949657 24268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:18.954797 24268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:18.955147 24268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:18.975982 24268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32903
I20260925 09:25:18.976080 24334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32903 every 8 connection(s)
I20260925 09:25:18.977613 24268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:18.982434 24335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:18.982810 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24268
I20260925 09:25:18.983003 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:18.983505 3400 external_mini_cluster.cc:1428] Setting key 510150a3520a30ed168f8fb44efdd7cc
I20260925 09:25:19.000973 24335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d: Bootstrap starting.
I20260925 09:25:19.001269 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:18.985516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59331 (local address 127.0.0.1:32903)
0925 09:25:18.986534 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:18.986551 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:18.986609 (+ 58us) server_negotiation.cc:407] Connection header received
0925 09:25:18.988641 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:18.988684 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:18.989429 (+ 745us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:18.990080 (+ 651us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:18.990787 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:18.993160 (+ 2373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:18.994876 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:18.996165 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:18.996421 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:18.997862 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:18.997876 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:18.997913 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:18.998775 (+ 862us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:18.998902 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:18.998940 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:18.999090 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:18.999357 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 09:25:18.999688 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":466,"threads_started":1}
I20260925 09:25:19.005018 24335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:19.005986 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:19.006417 24335 log.cc:824] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:19.010540 24335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d: No bootstrap required, opened a new log
I20260925 09:25:19.020284 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:19.011981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35141 (local address 127.0.0.1:32903)
0925 09:25:19.012217 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:19.012227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:19.012791 (+ 564us) server_negotiation.cc:407] Connection header received
0925 09:25:19.013029 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:19.013034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:19.013114 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:19.013214 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:19.013937 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:19.014794 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:19.017011 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:19.017548 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:19.017676 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:19.019173 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:19.019184 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:19.019190 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:19.019221 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:19.019293 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:19.019315 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:19.019415 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:19.019582 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 09:25:19.019766 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:25:19.022287 24335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826ff26eee234c809931ba29aa3d745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32903 } }
I20260925 09:25:19.022789 24335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:19.022936 24335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 826ff26eee234c809931ba29aa3d745d, State: Initialized, Role: FOLLOWER
I20260925 09:25:19.023590 24335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826ff26eee234c809931ba29aa3d745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32903 } }
I20260925 09:25:19.023861 24335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:19.024019 24335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:19.024209 24335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:19.026984 24335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826ff26eee234c809931ba29aa3d745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32903 } }
I20260925 09:25:19.027573 24335 leader_election.cc:302] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 826ff26eee234c809931ba29aa3d745d; no voters:
I20260925 09:25:19.028232 24335 leader_election.cc:288] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:19.028425 24345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:19.029281 24345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [term 1 LEADER]: Becoming Leader. State: Replica: 826ff26eee234c809931ba29aa3d745d, State: Running, Role: LEADER
I20260925 09:25:19.030263 24345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826ff26eee234c809931ba29aa3d745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32903 } }
I20260925 09:25:19.030948 24335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:19.033964 24346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "826ff26eee234c809931ba29aa3d745d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826ff26eee234c809931ba29aa3d745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32903 } } }
I20260925 09:25:19.034667 24346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:19.034017 24347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 826ff26eee234c809931ba29aa3d745d. Latest consensus state: current_term: 1 leader_uuid: "826ff26eee234c809931ba29aa3d745d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "826ff26eee234c809931ba29aa3d745d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32903 } } }
I20260925 09:25:19.035471 24347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:19.038317 24351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:19.045183 24351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:19.055608 24351 catalog_manager.cc:1347] Generated new cluster ID: 86aeb99216614ddcbc4e07a08c68534f
I20260925 09:25:19.055759 24351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:19.090236 24351 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:19.092247 24351 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:19.115167 24351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 826ff26eee234c809931ba29aa3d745d: Generated new TSK 0
I20260925 09:25:19.116118 24351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:19.130872 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24268
W20260925 09:25:19.132508 24330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59331
W20260925 09:25:19.133340 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24268: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:19.133528 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24268
2026-09-25T09:25:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:19 dist-test-slave-4wm5 krb5kdc[24366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:19 dist-test-slave-4wm5 krb5kdc[24366](info): set up 2 sockets
Sep 25 09:25:19 dist-test-slave-4wm5 krb5kdc[24366](info): commencing operation
krb5kdc: starting...
W20260925 09:25:20.262550 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:20 dist-test-slave-4wm5 krb5kdc[24366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:20.550477 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40061
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:20.898751 24379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:20.899075 24379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:20.899281 24379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:20.906932 24379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:20.907066 24379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:20.907151 24379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:20.907255 24379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:20.907325 24379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:20.917999 24379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:20.919363 24379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:20.920702 24379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:20.933588 24384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:20.933678 24387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:20.933614 24385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:20.934859 24379 server_base.cc:1034] running on GCE node
I20260925 09:25:20.935737 24379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:20.937198 24379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:20.938431 24379 hybrid_clock.cc:648] HybridClock initialized: now 1790328320938379 us; error 50 us; skew 500 ppm
I20260925 09:25:20.938827 24379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:20 dist-test-slave-4wm5 krb5kdc[24366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328320, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:20.942196 24379 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:20.943835 24379 webserver.cc:460] Webserver started at http://127.0.0.1:39967/ using document root <none> and password file <none>
I20260925 09:25:20.944555 24379 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:20.944658 24379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:20.944916 24379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:20.948122 24379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "59a9930e86dd44be9b66cac9d3ed7361"
format_stamp: "Formatted at 2026-09-25 09:25:20 on dist-test-slave-4wm5"
server_key: "466fff745828a5ac3d26acbd7aa2fa9b"
server_key_iv: "82b54159607583faca856ab492d5fa69"
server_key_version: "encryptionkey@0"
I20260925 09:25:20.948912 24379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "59a9930e86dd44be9b66cac9d3ed7361"
format_stamp: "Formatted at 2026-09-25 09:25:20 on dist-test-slave-4wm5"
server_key: "466fff745828a5ac3d26acbd7aa2fa9b"
server_key_iv: "82b54159607583faca856ab492d5fa69"
server_key_version: "encryptionkey@0"
I20260925 09:25:20.954926 24379 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:25:20.958806 24394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:20.960435 24379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:25:20.960644 24379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "59a9930e86dd44be9b66cac9d3ed7361"
format_stamp: "Formatted at 2026-09-25 09:25:20 on dist-test-slave-4wm5"
server_key: "466fff745828a5ac3d26acbd7aa2fa9b"
server_key_iv: "82b54159607583faca856ab492d5fa69"
server_key_version: "encryptionkey@0"
I20260925 09:25:20.960856 24379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:20.967368 24379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:20.972194 24379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:20.972514 24379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:20.992120 24379 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40061
I20260925 09:25:20.992161 24445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40061 every 8 connection(s)
I20260925 09:25:20.993698 24379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:20.997754 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24379
I20260925 09:25:20.998018 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:20.998265 24446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:20.998571 3400 external_mini_cluster.cc:1428] Setting key 6c45d55e72028f86170c86975088d0b1
I20260925 09:25:21.008183 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:21.000551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47585 (local address 127.0.0.1:40061)
0925 09:25:21.001456 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:21.001478 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:21.001539 (+ 61us) server_negotiation.cc:407] Connection header received
0925 09:25:21.003666 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:21.003711 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:21.004238 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:21.004764 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:21.006075 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:21.006083 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:21.006101 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:21.006121 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:21.006198 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:21.006228 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:21.006475 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:21.006604 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 09:25:21.006837 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":375,"threads_started":1}
I20260925 09:25:21.012755 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:21.013986 24446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361: Bootstrap starting.
I20260925 09:25:21.018456 24446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:21.019979 24446 log.cc:824] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:21.021168 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:21.018453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55903 (local address 127.0.0.1:40061)
0925 09:25:21.018643 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:21.018652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:21.018994 (+ 342us) server_negotiation.cc:407] Connection header received
0925 09:25:21.019134 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:21.019140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:21.019206 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:21.019316 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:21.020408 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:21.020413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:21.020418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:21.020438 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:21.020492 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:21.020521 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:21.020671 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:21.020816 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 09:25:21.020866 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 09:25:21.023598 24446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361: No bootstrap required, opened a new log
I20260925 09:25:21.031406 24446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a9930e86dd44be9b66cac9d3ed7361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40061 } }
I20260925 09:25:21.031816 24446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:21.031947 24446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a9930e86dd44be9b66cac9d3ed7361, State: Initialized, Role: FOLLOWER
I20260925 09:25:21.032624 24446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a9930e86dd44be9b66cac9d3ed7361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40061 } }
I20260925 09:25:21.032879 24446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:21.033038 24446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:21.033205 24446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:21.035863 24446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a9930e86dd44be9b66cac9d3ed7361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40061 } }
I20260925 09:25:21.036437 24446 leader_election.cc:302] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59a9930e86dd44be9b66cac9d3ed7361; no voters:
I20260925 09:25:21.037072 24446 leader_election.cc:288] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:21.037235 24456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:21.037971 24456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [term 1 LEADER]: Becoming Leader. State: Replica: 59a9930e86dd44be9b66cac9d3ed7361, State: Running, Role: LEADER
I20260925 09:25:21.038580 24456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a9930e86dd44be9b66cac9d3ed7361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40061 } }
I20260925 09:25:21.039261 24446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:21.041641 24457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59a9930e86dd44be9b66cac9d3ed7361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a9930e86dd44be9b66cac9d3ed7361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40061 } } }
I20260925 09:25:21.042670 24457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:21.041886 24458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59a9930e86dd44be9b66cac9d3ed7361. Latest consensus state: current_term: 1 leader_uuid: "59a9930e86dd44be9b66cac9d3ed7361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a9930e86dd44be9b66cac9d3ed7361" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40061 } } }
I20260925 09:25:21.043267 24458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:21.047223 24463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:21.054795 24463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:21.066912 24463 catalog_manager.cc:1347] Generated new cluster ID: b12ce97ce45446748aa6105ea61f914c
I20260925 09:25:21.067061 24463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:21.079066 24463 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:21.079576 24463 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:21.085975 24463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59a9930e86dd44be9b66cac9d3ed7361: Generated new TSK 0
I20260925 09:25:21.087154 24463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:21.095602 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24379
W20260925 09:25:21.096933 24441 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47585
W20260925 09:25:21.098140 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24379: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:21.098409 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24379
2026-09-25T09:25:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1962 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:21 dist-test-slave-4wm5 krb5kdc[24477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:21 dist-test-slave-4wm5 krb5kdc[24477](info): set up 2 sockets
Sep 25 09:25:21 dist-test-slave-4wm5 krb5kdc[24477](info): commencing operation
krb5kdc: starting...
W20260925 09:25:22.250469 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:22 dist-test-slave-4wm5 krb5kdc[24477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:22.541397 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33047
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:22.897727 24490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:22.898186 24490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:22.898360 24490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:22.906081 24490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:22.906245 24490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:22.906342 24490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:22.906447 24490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:22.906529 24490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:22.917044 24490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:22.918570 24490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:22.919934 24490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:22.932674 24495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:22.932651 24498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:22.932624 24496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:22.935894 24490 server_base.cc:1034] running on GCE node
I20260925 09:25:22.937002 24490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:22.939015 24490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:22.940311 24490 hybrid_clock.cc:648] HybridClock initialized: now 1790328322940260 us; error 63 us; skew 500 ppm
I20260925 09:25:22.940778 24490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:22 dist-test-slave-4wm5 krb5kdc[24477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:22.949726 24490 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:22.952051 24490 webserver.cc:460] Webserver started at http://127.0.0.1:43437/ using document root <none> and password file <none>
I20260925 09:25:22.952970 24490 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:22.953146 24490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:22.953497 24490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:22.957294 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "0fcd1b944c414fdf9700a0b9dc2842ad"
format_stamp: "Formatted at 2026-09-25 09:25:22 on dist-test-slave-4wm5"
server_key: "eb377913076368560098b1bc9f2db112"
server_key_iv: "68f4835c7221ab4b2fcbab5d00fdf89b"
server_key_version: "encryptionkey@0"
I20260925 09:25:22.958405 24490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "0fcd1b944c414fdf9700a0b9dc2842ad"
format_stamp: "Formatted at 2026-09-25 09:25:22 on dist-test-slave-4wm5"
server_key: "eb377913076368560098b1bc9f2db112"
server_key_iv: "68f4835c7221ab4b2fcbab5d00fdf89b"
server_key_version: "encryptionkey@0"
I20260925 09:25:22.964905 24490 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:25:22.969458 24505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:22.971313 24490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:22.971513 24490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "0fcd1b944c414fdf9700a0b9dc2842ad"
format_stamp: "Formatted at 2026-09-25 09:25:22 on dist-test-slave-4wm5"
server_key: "eb377913076368560098b1bc9f2db112"
server_key_iv: "68f4835c7221ab4b2fcbab5d00fdf89b"
server_key_version: "encryptionkey@0"
I20260925 09:25:22.971717 24490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:23.001039 24490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:23.006403 24490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:23.006718 24490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:23.027730 24490 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33047
I20260925 09:25:23.027747 24556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33047 every 8 connection(s)
I20260925 09:25:23.029213 24490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:23.031085 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24490
I20260925 09:25:23.031320 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:23.031798 3400 external_mini_cluster.cc:1428] Setting key c11d53392d49427c2ab29b96b5079b38
I20260925 09:25:23.034180 24557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:25:23 dist-test-slave-4wm5 krb5kdc[24477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:25:23.049616 24557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad: Bootstrap starting.
I20260925 09:25:23.053211 24557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:23.054430 24557 log.cc:824] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:23.058327 24557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad: No bootstrap required, opened a new log
I20260925 09:25:23.059083 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:23.034005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:43679 (local address 127.0.0.1:33047)
0925 09:25:23.034941 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:23.034954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:23.034997 (+ 43us) server_negotiation.cc:407] Connection header received
0925 09:25:23.036792 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:23.036817 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:23.037278 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:23.037748 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:23.039687 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:23.041224 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:23.043308 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:23.044192 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:23.044363 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:23.049934 (+ 5571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:23.049945 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:23.049963 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:23.050016 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:23.053601 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:23.054712 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:23.054720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:23.054728 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:23.054974 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:23.055604 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:23.055616 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:23.055620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:23.056334 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:23.056640 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:23.057429 (+ 789us) server_negotiation.cc:299] Negotiation successful
0925 09:25:23.057804 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":444,"threads_started":1}
I20260925 09:25:23.063611 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:23.067188 24557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33047 } }
I20260925 09:25:23.067644 24557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:23.067759 24557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcd1b944c414fdf9700a0b9dc2842ad, State: Initialized, Role: FOLLOWER
I20260925 09:25:23.068389 24557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33047 } }
I20260925 09:25:23.068984 24557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:23.069253 24557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:23.069474 24557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:23.072715 24557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33047 } }
I20260925 09:25:23.073294 24557 leader_election.cc:302] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fcd1b944c414fdf9700a0b9dc2842ad; no voters:
I20260925 09:25:23.074051 24557 leader_election.cc:288] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:23.074169 24567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:23.075570 24567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [term 1 LEADER]: Becoming Leader. State: Replica: 0fcd1b944c414fdf9700a0b9dc2842ad, State: Running, Role: LEADER
I20260925 09:25:23.076263 24567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33047 } }
I20260925 09:25:23.076673 24557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:23.081215 24568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33047 } } }
I20260925 09:25:23.081777 24568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:23.082545 24569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcd1b944c414fdf9700a0b9dc2842ad. Latest consensus state: current_term: 1 leader_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd1b944c414fdf9700a0b9dc2842ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33047 } } }
I20260925 09:25:23.082923 24569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:23.090791 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:23.068792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:36117 (local address 127.0.0.1:33047)
0925 09:25:23.069056 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:23.069063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:23.069449 (+ 386us) server_negotiation.cc:407] Connection header received
0925 09:25:23.069698 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:23.069706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:23.069808 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:23.069982 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:23.071955 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:23.072802 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:23.074825 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:23.075589 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:23.075719 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:23.079621 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:23.079629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:23.079634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:23.079690 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:23.084460 (+ 4770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:23.086108 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:23.086116 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:23.086122 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:23.086373 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:23.087201 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:23.087208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:23.087214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:23.087749 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:23.089610 (+ 1861us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:23.089858 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 09:25:23.090140 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 09:25:23.093014 24574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:23.100576 24574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:23.115377 24574 catalog_manager.cc:1347] Generated new cluster ID: eea503f464f945f1ae92cdf633f5ef5b
I20260925 09:25:23.115773 24574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:23.133461 24574 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:23.135524 24574 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:23.144469 24574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fcd1b944c414fdf9700a0b9dc2842ad: Generated new TSK 0
I20260925 09:25:23.145778 24574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:23.162411 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24490
I20260925 09:25:23.163846 24552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:23.259289 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24490
W20260925 09:25:23.278117 24475 connection.cc:535] client connection to 127.0.0.1:33047 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33047): Connection reset by peer (error 104)
2026-09-25T09:25:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2160 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:23 dist-test-slave-4wm5 krb5kdc[24590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:23 dist-test-slave-4wm5 krb5kdc[24590](info): set up 2 sockets
Sep 25 09:25:23 dist-test-slave-4wm5 krb5kdc[24590](info): commencing operation
krb5kdc: starting...
W20260925 09:25:24.402549 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:24 dist-test-slave-4wm5 krb5kdc[24590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:24.694416 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:25.086323 24603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:25.086717 24603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:25.086889 24603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:25.095614 24603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:25.095780 24603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:25.095860 24603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:25.095984 24603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:25.096053 24603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:25.108493 24603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:25.110183 24603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:25.111752 24603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:25.126752 24609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:25.126658 24611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:25.127035 24608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:25.128808 24603 server_base.cc:1034] running on GCE node
I20260925 09:25:25.130125 24603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:25.131965 24603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:25.133285 24603 hybrid_clock.cc:648] HybridClock initialized: now 1790328325133226 us; error 53 us; skew 500 ppm
I20260925 09:25:25.133941 24603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:25 dist-test-slave-4wm5 krb5kdc[24590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:25.137912 24603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:25.139747 24603 webserver.cc:460] Webserver started at http://127.0.0.1:40301/ using document root <none> and password file <none>
I20260925 09:25:25.140537 24603 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:25.140652 24603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:25.140928 24603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:25.146116 24603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "f614742b73354f29ab64e8a166386abe"
format_stamp: "Formatted at 2026-09-25 09:25:25 on dist-test-slave-4wm5"
server_key: "26d1a6db91949731c6a24de84d34a4ef"
server_key_iv: "611f1964e69d3b87898e64878bf2b0b8"
server_key_version: "encryptionkey@0"
I20260925 09:25:25.147504 24603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "f614742b73354f29ab64e8a166386abe"
format_stamp: "Formatted at 2026-09-25 09:25:25 on dist-test-slave-4wm5"
server_key: "26d1a6db91949731c6a24de84d34a4ef"
server_key_iv: "611f1964e69d3b87898e64878bf2b0b8"
server_key_version: "encryptionkey@0"
I20260925 09:25:25.154403 24603 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:25:25.159179 24618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:25.161192 24603 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260925 09:25:25.161530 24603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "f614742b73354f29ab64e8a166386abe"
format_stamp: "Formatted at 2026-09-25 09:25:25 on dist-test-slave-4wm5"
server_key: "26d1a6db91949731c6a24de84d34a4ef"
server_key_iv: "611f1964e69d3b87898e64878bf2b0b8"
server_key_version: "encryptionkey@0"
I20260925 09:25:25.161813 24603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:25.196988 24603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:25.202266 24603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:25.202612 24603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:25.223286 24603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43243
I20260925 09:25:25.223325 24669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43243 every 8 connection(s)
I20260925 09:25:25.224889 24603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:25.229127 24670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:25.234580 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24603
I20260925 09:25:25.234808 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:25.235256 3400 external_mini_cluster.cc:1428] Setting key 0cfb8cf1bbbebd1bec8867c2671e8ec5
I20260925 09:25:25.243711 24670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe: Bootstrap starting.
I20260925 09:25:25.248126 24670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:25.249752 24670 log.cc:824] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:25.253968 24670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe: No bootstrap required, opened a new log
I20260925 09:25:25.254326 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:25.237530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:57421 (local address 127.0.0.1:43243)
0925 09:25:25.238531 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:25.238554 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:25.238614 (+ 60us) server_negotiation.cc:407] Connection header received
0925 09:25:25.241741 (+ 3127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:25.241784 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:25.242528 (+ 744us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:25.243082 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:25.243773 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:25.245957 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:25.247763 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:25.249015 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:25.249245 (+ 230us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:25.250911 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:25.250920 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:25.250938 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:25.251860 (+ 922us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:25.251972 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:25.252007 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:25.252213 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:25.252642 (+ 429us) server_negotiation.cc:299] Negotiation successful
0925 09:25:25.252850 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":443,"threads_started":1}
I20260925 09:25:25.259593 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:25.263346 24670 raft_consensus.cc:348] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f614742b73354f29ab64e8a166386abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260925 09:25:25.263928 24670 raft_consensus.cc:374] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:25.264029 24670 raft_consensus.cc:729] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f614742b73354f29ab64e8a166386abe, State: Initialized, Role: FOLLOWER
I20260925 09:25:25.264816 24670 consensus_queue.cc:260] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f614742b73354f29ab64e8a166386abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260925 09:25:25.265131 24670 raft_consensus.cc:388] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:25.265301 24670 raft_consensus.cc:482] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:25.265535 24670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:25.268887 24670 raft_consensus.cc:504] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f614742b73354f29ab64e8a166386abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260925 09:25:25.269652 24670 leader_election.cc:302] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f614742b73354f29ab64e8a166386abe; no voters:
I20260925 09:25:25.270524 24670 leader_election.cc:288] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:25.270641 24680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:25.273066 24680 raft_consensus.cc:686] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [term 1 LEADER]: Becoming Leader. State: Replica: f614742b73354f29ab64e8a166386abe, State: Running, Role: LEADER
I20260925 09:25:25.274130 24670 sys_catalog.cc:564] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:25.274065 24680 consensus_queue.cc:237] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f614742b73354f29ab64e8a166386abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260925 09:25:25.274318 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:25.264692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:47737 (local address 127.0.0.1:43243)
0925 09:25:25.264912 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:25.264919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:25.265507 (+ 588us) server_negotiation.cc:407] Connection header received
0925 09:25:25.265747 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:25.265762 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:25.265895 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:25.266023 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:25.266584 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:25.267571 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:25.269619 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:25.270411 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:25.270594 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:25.272751 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:25.272764 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:25.272770 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:25.272821 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:25.272966 (+ 145us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:25.272999 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:25.273195 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:25.273753 (+ 558us) server_negotiation.cc:299] Negotiation successful
0925 09:25:25.273819 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 09:25:25.283351 24682 sys_catalog.cc:455] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader f614742b73354f29ab64e8a166386abe. Latest consensus state: current_term: 1 leader_uuid: "f614742b73354f29ab64e8a166386abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f614742b73354f29ab64e8a166386abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } } }
I20260925 09:25:25.283398 24681 sys_catalog.cc:455] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f614742b73354f29ab64e8a166386abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f614742b73354f29ab64e8a166386abe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } } }
I20260925 09:25:25.283820 24682 sys_catalog.cc:458] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:25.283885 24681 sys_catalog.cc:458] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:25.284893 24688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:25.291991 24688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:25.302157 24688 catalog_manager.cc:1347] Generated new cluster ID: e1322fdb768a4d66bbed73409ad7caff
I20260925 09:25:25.302309 24688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:25.311412 24688 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:25.312776 24688 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:25.330000 24688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f614742b73354f29ab64e8a166386abe: Generated new TSK 0
I20260925 09:25:25.331228 24688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:25.347411 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24603
W20260925 09:25:25.348845 24665 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:57421
W20260925 09:25:25.349802 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24603: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:25.350023 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24603
W20260925 09:25:25.367774 24588 connection.cc:535] client connection to 127.0.0.1:43243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43243): Connection reset by peer (error 104)
2026-09-25T09:25:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2089 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:25 dist-test-slave-4wm5 krb5kdc[24701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:25 dist-test-slave-4wm5 krb5kdc[24701](info): set up 2 sockets
Sep 25 09:25:25 dist-test-slave-4wm5 krb5kdc[24701](info): commencing operation
krb5kdc: starting...
W20260925 09:25:26.486670 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:26 dist-test-slave-4wm5 krb5kdc[24701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:26.774230 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39247
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:27.133204 24714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:27.133780 24714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:27.134040 24714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:27.148731 24714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:27.148999 24714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:27.149123 24714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:27.149228 24714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:27.149304 24714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:27.167930 24714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:27.169307 24714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:27.170799 24714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:27.183781 24722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:27.183861 24720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:27.183812 24719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:27.188060 24714 server_base.cc:1034] running on GCE node
I20260925 09:25:27.189045 24714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:27.190717 24714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:27.191987 24714 hybrid_clock.cc:648] HybridClock initialized: now 1790328327191940 us; error 47 us; skew 500 ppm
I20260925 09:25:27.192363 24714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:27 dist-test-slave-4wm5 krb5kdc[24701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328327, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:27.195812 24714 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:27.197533 24714 webserver.cc:460] Webserver started at http://127.0.0.1:35493/ using document root <none> and password file <none>
I20260925 09:25:27.198325 24714 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:27.198446 24714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:27.198705 24714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:27.201964 24714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "5559ceb7ae654aecb049534757c0b72b"
format_stamp: "Formatted at 2026-09-25 09:25:27 on dist-test-slave-4wm5"
server_key: "5edbc8ff89638d7b515183731293eba5"
server_key_iv: "f9482539b6305e0d16c0acfe5d2d3e1d"
server_key_version: "encryptionkey@0"
I20260925 09:25:27.202769 24714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "5559ceb7ae654aecb049534757c0b72b"
format_stamp: "Formatted at 2026-09-25 09:25:27 on dist-test-slave-4wm5"
server_key: "5edbc8ff89638d7b515183731293eba5"
server_key_iv: "f9482539b6305e0d16c0acfe5d2d3e1d"
server_key_version: "encryptionkey@0"
I20260925 09:25:27.208513 24714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260925 09:25:27.212477 24729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:27.214174 24714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:25:27.214354 24714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "5559ceb7ae654aecb049534757c0b72b"
format_stamp: "Formatted at 2026-09-25 09:25:27 on dist-test-slave-4wm5"
server_key: "5edbc8ff89638d7b515183731293eba5"
server_key_iv: "f9482539b6305e0d16c0acfe5d2d3e1d"
server_key_version: "encryptionkey@0"
I20260925 09:25:27.214541 24714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:27.220767 24714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:27.225396 24714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:27.225666 24714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:27.246934 24714 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39247
I20260925 09:25:27.247085 24780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39247 every 8 connection(s)
I20260925 09:25:27.248656 24714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:27.252570 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24714
I20260925 09:25:27.252805 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:27.253372 3400 external_mini_cluster.cc:1428] Setting key 74f1e2d5a349a7517b7ba95938b9c18f
I20260925 09:25:27.253752 24781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:27.263554 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:27.255578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:38259 (local address 127.0.0.1:39247)
0925 09:25:27.256486 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:27.256503 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:27.256555 (+ 52us) server_negotiation.cc:407] Connection header received
0925 09:25:27.259065 (+ 2510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:27.259099 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:27.259642 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:27.260201 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:27.261315 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:27.261323 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:27.261342 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:27.261361 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:27.261451 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:27.261479 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:27.261734 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:27.261930 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 09:25:27.262167 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":372,"threads_started":1}
I20260925 09:25:27.268206 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:27.270046 24781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b: Bootstrap starting.
I20260925 09:25:27.273710 24781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:27.275101 24781 log.cc:824] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:27.276065 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:27.273142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:60057 (local address 127.0.0.1:39247)
0925 09:25:27.273331 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:27.273336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:27.273721 (+ 385us) server_negotiation.cc:407] Connection header received
0925 09:25:27.273962 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:27.273970 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:27.274037 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:27.274174 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:27.275167 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:27.275172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:27.275176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:27.275194 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:27.275245 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:27.275267 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:27.275387 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:27.275579 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 09:25:27.275649 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 09:25:27.278738 24781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b: No bootstrap required, opened a new log
I20260925 09:25:27.286779 24781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5559ceb7ae654aecb049534757c0b72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260925 09:25:27.287180 24781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:27.287256 24781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5559ceb7ae654aecb049534757c0b72b, State: Initialized, Role: FOLLOWER
I20260925 09:25:27.287914 24781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5559ceb7ae654aecb049534757c0b72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260925 09:25:27.288121 24781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:27.288225 24781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:27.288364 24781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:27.291180 24781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5559ceb7ae654aecb049534757c0b72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260925 09:25:27.291662 24781 leader_election.cc:302] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5559ceb7ae654aecb049534757c0b72b; no voters:
I20260925 09:25:27.292244 24781 leader_election.cc:288] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:27.292376 24791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:27.293172 24791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [term 1 LEADER]: Becoming Leader. State: Replica: 5559ceb7ae654aecb049534757c0b72b, State: Running, Role: LEADER
I20260925 09:25:27.293933 24791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5559ceb7ae654aecb049534757c0b72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260925 09:25:27.294838 24781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:27.296947 24792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5559ceb7ae654aecb049534757c0b72b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5559ceb7ae654aecb049534757c0b72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } } }
I20260925 09:25:27.297418 24792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:27.297092 24793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5559ceb7ae654aecb049534757c0b72b. Latest consensus state: current_term: 1 leader_uuid: "5559ceb7ae654aecb049534757c0b72b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5559ceb7ae654aecb049534757c0b72b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } } }
I20260925 09:25:27.297847 24793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:27.299578 24799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:27.305320 24799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:27.315267 24799 catalog_manager.cc:1347] Generated new cluster ID: 214472b251854413a9b4052299c5bf3d
I20260925 09:25:27.315418 24799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:27.349589 24799 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:27.350245 24799 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:27.366128 24799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5559ceb7ae654aecb049534757c0b72b: Generated new TSK 0
I20260925 09:25:27.367102 24799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:27.380996 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24714
W20260925 09:25:27.382222 24776 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:38259
W20260925 09:25:27.383034 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24714: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:27.383231 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24714
2026-09-25T09:25:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:27 dist-test-slave-4wm5 krb5kdc[24812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:27 dist-test-slave-4wm5 krb5kdc[24812](info): set up 2 sockets
Sep 25 09:25:27 dist-test-slave-4wm5 krb5kdc[24812](info): commencing operation
krb5kdc: starting...
W20260925 09:25:28.542426 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:28 dist-test-slave-4wm5 krb5kdc[24812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:28.837977 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:29.202430 24825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:29.202766 24825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:29.202972 24825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:29.210419 24825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:29.210546 24825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:29.210640 24825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:29.210740 24825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:29.210829 24825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:29.221055 24825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:29.222493 24825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:29.223924 24825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:29.236753 24830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:29.236667 24833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:29.236682 24831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:29.237916 24825 server_base.cc:1034] running on GCE node
I20260925 09:25:29.238824 24825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:29.240542 24825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:29.241760 24825 hybrid_clock.cc:648] HybridClock initialized: now 1790328329241703 us; error 52 us; skew 500 ppm
I20260925 09:25:29.242190 24825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:29 dist-test-slave-4wm5 krb5kdc[24812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:29.251327 24825 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:29.253178 24825 webserver.cc:460] Webserver started at http://127.0.0.1:39763/ using document root <none> and password file <none>
I20260925 09:25:29.254025 24825 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:29.254180 24825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:29.254529 24825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:29.257786 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "afd5013858454c998fb38958573677bd"
format_stamp: "Formatted at 2026-09-25 09:25:29 on dist-test-slave-4wm5"
server_key: "48940b91a6c0080a211bee2c9a7dca28"
server_key_iv: "61cddc790ccd3628eeab9b7047df7bc3"
server_key_version: "encryptionkey@0"
I20260925 09:25:29.258776 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "afd5013858454c998fb38958573677bd"
format_stamp: "Formatted at 2026-09-25 09:25:29 on dist-test-slave-4wm5"
server_key: "48940b91a6c0080a211bee2c9a7dca28"
server_key_iv: "61cddc790ccd3628eeab9b7047df7bc3"
server_key_version: "encryptionkey@0"
I20260925 09:25:29.264627 24825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 09:25:29.268651 24840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:29.270582 24825 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260925 09:25:29.270824 24825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "afd5013858454c998fb38958573677bd"
format_stamp: "Formatted at 2026-09-25 09:25:29 on dist-test-slave-4wm5"
server_key: "48940b91a6c0080a211bee2c9a7dca28"
server_key_iv: "61cddc790ccd3628eeab9b7047df7bc3"
server_key_version: "encryptionkey@0"
I20260925 09:25:29.271075 24825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:29.288075 24825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:29.293215 24825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:29.293545 24825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:29.314369 24825 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36531
I20260925 09:25:29.314375 24891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36531 every 8 connection(s)
I20260925 09:25:29.315920 24825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:29.317022 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24825
I20260925 09:25:29.317214 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:29.317723 3400 external_mini_cluster.cc:1428] Setting key 62be21bb8cea22200b31c406b057e002
I20260925 09:25:29.320935 24892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:25:29 dist-test-slave-4wm5 krb5kdc[24812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 09:25:29.335022 24892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd: Bootstrap starting.
I20260925 09:25:29.339668 24892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:29.340945 24892 log.cc:824] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:29.344743 24892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd: No bootstrap required, opened a new log
I20260925 09:25:29.344786 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:29.319767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:50583 (local address 127.0.0.1:36531)
0925 09:25:29.320600 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:29.320616 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:29.320667 (+ 51us) server_negotiation.cc:407] Connection header received
0925 09:25:29.323030 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:29.323061 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:29.323644 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:29.324187 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:29.325712 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:29.327940 (+ 2228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:29.329365 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:29.330450 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:29.330638 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:29.335204 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:29.335218 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:29.335240 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:29.335301 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:29.339395 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:29.340367 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:29.340377 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:29.340386 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:29.340601 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:29.341343 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:29.341349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:29.341352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:29.342105 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:29.342457 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:29.343015 (+ 558us) server_negotiation.cc:299] Negotiation successful
0925 09:25:29.343381 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":330,"threads_started":1}
I20260925 09:25:29.349910 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:29.354291 24892 raft_consensus.cc:348] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd5013858454c998fb38958573677bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36531 } }
I20260925 09:25:29.355078 24892 raft_consensus.cc:374] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:29.355201 24892 raft_consensus.cc:729] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afd5013858454c998fb38958573677bd, State: Initialized, Role: FOLLOWER
I20260925 09:25:29.355883 24892 consensus_queue.cc:260] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd5013858454c998fb38958573677bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36531 } }
I20260925 09:25:29.356149 24892 raft_consensus.cc:388] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:29.356312 24892 raft_consensus.cc:482] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:29.356492 24892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:29.358963 24892 raft_consensus.cc:504] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd5013858454c998fb38958573677bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36531 } }
I20260925 09:25:29.359553 24892 leader_election.cc:302] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afd5013858454c998fb38958573677bd; no voters:
I20260925 09:25:29.360311 24892 leader_election.cc:288] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:29.360469 24902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:29.362114 24902 raft_consensus.cc:686] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [term 1 LEADER]: Becoming Leader. State: Replica: afd5013858454c998fb38958573677bd, State: Running, Role: LEADER
I20260925 09:25:29.362762 24902 consensus_queue.cc:237] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd5013858454c998fb38958573677bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36531 } }
I20260925 09:25:29.363478 24892 sys_catalog.cc:564] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:29.364984 24902 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afd5013858454c998fb38958573677bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd5013858454c998fb38958573677bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36531 } } }
I20260925 09:25:29.366398 24902 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:29.373941 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:29.354684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:52021 (local address 127.0.0.1:36531)
0925 09:25:29.354884 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:29.354892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:29.355358 (+ 466us) server_negotiation.cc:407] Connection header received
0925 09:25:29.355693 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:29.355697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:29.355765 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:29.355882 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:29.357514 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:29.358403 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:29.359910 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:29.360502 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:29.360661 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:29.364105 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:29.364113 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:29.364120 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:29.364165 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:29.368186 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:29.369641 (+ 1455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:29.369650 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:29.369656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:29.369956 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:29.370688 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:29.370695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:29.370701 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:29.371210 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:29.371583 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:29.373083 (+ 1500us) server_negotiation.cc:299] Negotiation successful
0925 09:25:29.373345 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 09:25:29.366356 24903 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader afd5013858454c998fb38958573677bd. Latest consensus state: current_term: 1 leader_uuid: "afd5013858454c998fb38958573677bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd5013858454c998fb38958573677bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36531 } } }
I20260925 09:25:29.374542 24903 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:29.376579 24909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:29.382018 24909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:29.391235 24909 catalog_manager.cc:1347] Generated new cluster ID: 11bd7db7d55042ca85e0ce1c2df5f021
I20260925 09:25:29.391376 24909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:29.419711 24909 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:29.421747 24909 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:29.445087 24909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afd5013858454c998fb38958573677bd: Generated new TSK 0
I20260925 09:25:29.446164 24909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:29.461987 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24825
I20260925 09:25:29.463692 24887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:29.561141 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24825
W20260925 09:25:29.580097 24810 connection.cc:535] client connection to 127.0.0.1:36531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36531): Connection reset by peer (error 104)
2026-09-25T09:25:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2181 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:29 dist-test-slave-4wm5 krb5kdc[24924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:29 dist-test-slave-4wm5 krb5kdc[24924](info): set up 2 sockets
Sep 25 09:25:29 dist-test-slave-4wm5 krb5kdc[24924](info): commencing operation
krb5kdc: starting...
W20260925 09:25:30.690606 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:30 dist-test-slave-4wm5 krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:30.982184 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:31.343190 24937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:31.343529 24937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:31.343700 24937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:31.351573 24937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:31.351713 24937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:31.351794 24937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:31.351902 24937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:31.351990 24937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:31.362679 24937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:31.364106 24937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:31.365489 24937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:31.378710 24943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:31.378729 24945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:31.379019 24942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:31.379808 24937 server_base.cc:1034] running on GCE node
I20260925 09:25:31.380697 24937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:31.382476 24937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:31.383745 24937 hybrid_clock.cc:648] HybridClock initialized: now 1790328331383649 us; error 95 us; skew 500 ppm
I20260925 09:25:31.384166 24937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:31 dist-test-slave-4wm5 krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:31.387984 24937 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:31.389782 24937 webserver.cc:460] Webserver started at http://127.0.0.1:38715/ using document root <none> and password file <none>
I20260925 09:25:31.390627 24937 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:31.390797 24937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:31.391124 24937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:31.394387 24937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "17c65695e2c040c885818c04b93afe30"
format_stamp: "Formatted at 2026-09-25 09:25:31 on dist-test-slave-4wm5"
server_key: "499714e85048206c35b604d7c28fb29b"
server_key_iv: "866dc2172640ffcae2a4fdc708e1a0f8"
server_key_version: "encryptionkey@0"
I20260925 09:25:31.395205 24937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "17c65695e2c040c885818c04b93afe30"
format_stamp: "Formatted at 2026-09-25 09:25:31 on dist-test-slave-4wm5"
server_key: "499714e85048206c35b604d7c28fb29b"
server_key_iv: "866dc2172640ffcae2a4fdc708e1a0f8"
server_key_version: "encryptionkey@0"
I20260925 09:25:31.400985 24937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 09:25:31.405128 24952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:31.406677 24937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:25:31.406889 24937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "17c65695e2c040c885818c04b93afe30"
format_stamp: "Formatted at 2026-09-25 09:25:31 on dist-test-slave-4wm5"
server_key: "499714e85048206c35b604d7c28fb29b"
server_key_iv: "866dc2172640ffcae2a4fdc708e1a0f8"
server_key_version: "encryptionkey@0"
I20260925 09:25:31.407116 24937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:31.421530 24937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:31.427016 24937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:31.427373 24937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:31.451201 24937 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35821
I20260925 09:25:31.451238 25003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35821 every 8 connection(s)
I20260925 09:25:31.453104 24937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:31.457851 25004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:31.461575 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 24937
I20260925 09:25:31.461867 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:31.462407 3400 external_mini_cluster.cc:1428] Setting key 63bd3ec27a620a461f9c2efde8a598b1
I20260925 09:25:31.472038 25004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30: Bootstrap starting.
I20260925 09:25:31.475605 25004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:31.476825 25004 log.cc:824] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:31.479178 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:31.464863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:38041 (local address 127.0.0.1:35821)
0925 09:25:31.465996 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:31.466013 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:31.466077 (+ 64us) server_negotiation.cc:407] Connection header received
0925 09:25:31.468098 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:31.468127 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:31.468694 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:31.469192 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:31.469914 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:31.471435 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:31.473199 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:31.474147 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:31.474393 (+ 246us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:31.475952 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:31.475960 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:31.475977 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:31.476632 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:31.476730 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:31.476761 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:31.476957 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:31.477304 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 09:25:31.477569 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":446,"threads_started":1}
I20260925 09:25:31.480754 25004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30: No bootstrap required, opened a new log
I20260925 09:25:31.484508 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:31.489351 25004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c65695e2c040c885818c04b93afe30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35821 } }
I20260925 09:25:31.489933 25004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:31.490087 25004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c65695e2c040c885818c04b93afe30, State: Initialized, Role: FOLLOWER
I20260925 09:25:31.490906 25004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c65695e2c040c885818c04b93afe30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35821 } }
I20260925 09:25:31.491245 25004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:31.491434 25004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:31.491636 25004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:31.495064 25004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c65695e2c040c885818c04b93afe30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35821 } }
I20260925 09:25:31.495756 25004 leader_election.cc:302] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17c65695e2c040c885818c04b93afe30; no voters:
I20260925 09:25:31.496613 25004 leader_election.cc:288] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:31.496762 25014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:31.498349 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:31.490168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:45351 (local address 127.0.0.1:35821)
0925 09:25:31.490373 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:31.490378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:31.490824 (+ 446us) server_negotiation.cc:407] Connection header received
0925 09:25:31.491085 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:31.491089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:31.491168 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:31.491365 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:31.492049 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:31.492937 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:31.494937 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:31.495561 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:31.495673 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:31.497263 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:31.497274 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:31.497281 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:31.497309 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:31.497370 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:31.497392 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:31.497505 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:31.497893 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 09:25:31.497960 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 09:25:31.498281 25014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [term 1 LEADER]: Becoming Leader. State: Replica: 17c65695e2c040c885818c04b93afe30, State: Running, Role: LEADER
I20260925 09:25:31.499054 25014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c65695e2c040c885818c04b93afe30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35821 } }
I20260925 09:25:31.499478 25004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:31.502746 25015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17c65695e2c040c885818c04b93afe30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c65695e2c040c885818c04b93afe30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35821 } } }
I20260925 09:25:31.503213 25015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:31.503690 25016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17c65695e2c040c885818c04b93afe30. Latest consensus state: current_term: 1 leader_uuid: "17c65695e2c040c885818c04b93afe30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17c65695e2c040c885818c04b93afe30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35821 } } }
I20260925 09:25:31.504118 25016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:31.505478 25024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:31.511210 25024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:31.521282 25024 catalog_manager.cc:1347] Generated new cluster ID: d061809c902741c4a664f28c2bfb4e8e
I20260925 09:25:31.521492 25024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:31.540344 25024 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:31.542109 25024 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:31.552767 25024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17c65695e2c040c885818c04b93afe30: Generated new TSK 0
I20260925 09:25:31.553793 25024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:31.564097 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24937
W20260925 09:25:31.565536 24999 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:38041
W20260925 09:25:31.566489 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 24937: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:31.566639 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 24937
2026-09-25T09:25:31Z chronyd exiting
W20260925 09:25:31.582978 24922 connection.cc:535] client connection to 127.0.0.1:35821 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35821): Connection reset by peer (error 104)
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:31 dist-test-slave-4wm5 krb5kdc[25035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:31 dist-test-slave-4wm5 krb5kdc[25035](info): set up 2 sockets
Sep 25 09:25:31 dist-test-slave-4wm5 krb5kdc[25035](info): commencing operation
krb5kdc: starting...
W20260925 09:25:32.702630 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:32 dist-test-slave-4wm5 krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 09:25:32.999521 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:33.376078 25048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:33.376434 25048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:33.376616 25048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:33.384234 25048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:33.384385 25048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:33.384433 25048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:33.384568 25048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:33.384652 25048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:33.395409 25048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:33.396876 25048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:33.398309 25048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:33.412248 25056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:33.412333 25054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:33.412295 25053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:33.413130 25048 server_base.cc:1034] running on GCE node
I20260925 09:25:33.414191 25048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:33.415786 25048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:33.417057 25048 hybrid_clock.cc:648] HybridClock initialized: now 1790328333416988 us; error 59 us; skew 500 ppm
I20260925 09:25:33.417557 25048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:33 dist-test-slave-4wm5 krb5kdc[25035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:33.421579 25048 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 09:25:33.423657 25048 webserver.cc:460] Webserver started at http://127.0.0.1:37815/ using document root <none> and password file <none>
I20260925 09:25:33.424506 25048 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:33.424659 25048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:33.425011 25048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:33.428481 25048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "749d66f683f94c8183faaf3487bacaa8"
format_stamp: "Formatted at 2026-09-25 09:25:33 on dist-test-slave-4wm5"
server_key: "0c196bfd4113f2b7c180ae49f706fee9"
server_key_iv: "4b34640bb2f8087fb79a44b93197dbf9"
server_key_version: "encryptionkey@0"
I20260925 09:25:33.429401 25048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "749d66f683f94c8183faaf3487bacaa8"
format_stamp: "Formatted at 2026-09-25 09:25:33 on dist-test-slave-4wm5"
server_key: "0c196bfd4113f2b7c180ae49f706fee9"
server_key_iv: "4b34640bb2f8087fb79a44b93197dbf9"
server_key_version: "encryptionkey@0"
I20260925 09:25:33.435878 25048 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260925 09:25:33.440310 25063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:33.442190 25048 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:33.442457 25048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "749d66f683f94c8183faaf3487bacaa8"
format_stamp: "Formatted at 2026-09-25 09:25:33 on dist-test-slave-4wm5"
server_key: "0c196bfd4113f2b7c180ae49f706fee9"
server_key_iv: "4b34640bb2f8087fb79a44b93197dbf9"
server_key_version: "encryptionkey@0"
I20260925 09:25:33.442742 25048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:33.449669 25048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:33.455147 25048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:33.455511 25048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:33.476257 25048 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39801
I20260925 09:25:33.476293 25114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39801 every 8 connection(s)
I20260925 09:25:33.477886 25048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:33.478195 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25048
I20260925 09:25:33.478425 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:33.479176 3400 external_mini_cluster.cc:1428] Setting key 263341d76b39d89debaa8463dd2cd4c3
I20260925 09:25:33.482905 25115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:33.488013 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:33.481332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:40473 (local address 127.0.0.1:39801)
0925 09:25:33.482237 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:33.482250 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:33.482330 (+ 80us) server_negotiation.cc:407] Connection header received
0925 09:25:33.484080 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:33.484105 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:33.484477 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:33.484893 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:33.486098 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:33.486104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:33.486121 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:33.486135 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:33.486214 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:33.486234 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:33.486516 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:33.486656 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 09:25:33.486882 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":384,"threads_started":1}
I20260925 09:25:33.492353 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:33.495491 25115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8: Bootstrap starting.
I20260925 09:25:33.499625 25115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:33.500245 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:33.497204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:47279 (local address 127.0.0.1:39801)
0925 09:25:33.497448 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:33.497454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:33.497773 (+ 319us) server_negotiation.cc:407] Connection header received
0925 09:25:33.498018 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:33.498022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:33.498078 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:33.498172 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:33.499328 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:33.499333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:33.499336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:33.499353 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:33.499397 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:33.499412 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:33.499515 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:33.499705 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 09:25:33.499937 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260925 09:25:33.501039 25115 log.cc:824] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:33.504094 25115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8: No bootstrap required, opened a new log
I20260925 09:25:33.512274 25115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749d66f683f94c8183faaf3487bacaa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260925 09:25:33.512754 25115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:33.512871 25115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749d66f683f94c8183faaf3487bacaa8, State: Initialized, Role: FOLLOWER
I20260925 09:25:33.513504 25115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749d66f683f94c8183faaf3487bacaa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260925 09:25:33.513757 25115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:33.513947 25115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:33.514132 25115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:33.516709 25115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749d66f683f94c8183faaf3487bacaa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260925 09:25:33.517279 25115 leader_election.cc:302] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 749d66f683f94c8183faaf3487bacaa8; no voters:
I20260925 09:25:33.517997 25115 leader_election.cc:288] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:33.518162 25125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:33.518961 25125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [term 1 LEADER]: Becoming Leader. State: Replica: 749d66f683f94c8183faaf3487bacaa8, State: Running, Role: LEADER
I20260925 09:25:33.519555 25125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749d66f683f94c8183faaf3487bacaa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260925 09:25:33.520222 25115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:33.522437 25127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 749d66f683f94c8183faaf3487bacaa8. Latest consensus state: current_term: 1 leader_uuid: "749d66f683f94c8183faaf3487bacaa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749d66f683f94c8183faaf3487bacaa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } } }
I20260925 09:25:33.522776 25126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "749d66f683f94c8183faaf3487bacaa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749d66f683f94c8183faaf3487bacaa8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } } }
I20260925 09:25:33.523109 25126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:33.523118 25127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:33.524077 25133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:33.530220 25133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:33.542934 25133 catalog_manager.cc:1347] Generated new cluster ID: 04af181420f549a5b0a7377e08e5db11
I20260925 09:25:33.543085 25133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:33.556804 25133 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:33.557304 25133 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:33.572119 25133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 749d66f683f94c8183faaf3487bacaa8: Generated new TSK 0
I20260925 09:25:33.573118 25133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:33.577705 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25048
W20260925 09:25:33.578861 25110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.37:40473
W20260925 09:25:33.579623 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25048: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:33.579802 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25048
2026-09-25T09:25:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:33 dist-test-slave-4wm5 krb5kdc[25146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:33 dist-test-slave-4wm5 krb5kdc[25146](info): set up 2 sockets
Sep 25 09:25:33 dist-test-slave-4wm5 krb5kdc[25146](info): commencing operation
krb5kdc: starting...
W20260925 09:25:34.738443 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:34 dist-test-slave-4wm5 krb5kdc[25146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:35.047451 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43849
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:35.405169 25159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:35.405488 25159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:35.405694 25159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:35.413405 25159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:35.413547 25159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:35.413640 25159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:35.413758 25159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:35.413815 25159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:35.424351 25159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43849
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:35.425796 25159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:35.427219 25159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:35.440313 25164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:35.440297 25165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:35.440340 25167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:35.441695 25159 server_base.cc:1034] running on GCE node
I20260925 09:25:35.442624 25159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:35.444248 25159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:35.445554 25159 hybrid_clock.cc:648] HybridClock initialized: now 1790328335445492 us; error 45 us; skew 500 ppm
I20260925 09:25:35.446223 25159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:35 dist-test-slave-4wm5 krb5kdc[25146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328335, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:35.450039 25159 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:35.451763 25159 webserver.cc:460] Webserver started at http://127.3.82.62:41393/ using document root <none> and password file <none>
I20260925 09:25:35.452538 25159 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:35.452683 25159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:35.453019 25159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:35.456553 25159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "4b3dfcb185bc46278ec09c60e158b9cc"
format_stamp: "Formatted at 2026-09-25 09:25:35 on dist-test-slave-4wm5"
server_key: "7d7c02c385abb8177c6e9b1fb2db693c"
server_key_iv: "ec8aeb3fe1f6e282ab9d44cceec82722"
server_key_version: "encryptionkey@0"
I20260925 09:25:35.457428 25159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "4b3dfcb185bc46278ec09c60e158b9cc"
format_stamp: "Formatted at 2026-09-25 09:25:35 on dist-test-slave-4wm5"
server_key: "7d7c02c385abb8177c6e9b1fb2db693c"
server_key_iv: "ec8aeb3fe1f6e282ab9d44cceec82722"
server_key_version: "encryptionkey@0"
I20260925 09:25:35.463581 25159 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.005s
I20260925 09:25:35.468163 25174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:35.469933 25159 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:35.470166 25159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "4b3dfcb185bc46278ec09c60e158b9cc"
format_stamp: "Formatted at 2026-09-25 09:25:35 on dist-test-slave-4wm5"
server_key: "7d7c02c385abb8177c6e9b1fb2db693c"
server_key_iv: "ec8aeb3fe1f6e282ab9d44cceec82722"
server_key_version: "encryptionkey@0"
I20260925 09:25:35.470402 25159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:35.494091 25159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:35.498620 25159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:35.498934 25159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:35.521153 25159 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:43849
I20260925 09:25:35.521175 25225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:43849 every 8 connection(s)
I20260925 09:25:35.522742 25159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:35.526925 25226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:35.527096 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25159
I20260925 09:25:35.527321 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:35.527827 3400 external_mini_cluster.cc:1428] Setting key 575628e9af81923d5644b13598f14316
I20260925 09:25:35.541546 25226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc: Bootstrap starting.
Sep 25 09:25:35 dist-test-slave-4wm5 krb5kdc[25146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:35.545430 25226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:35.546832 25226 log.cc:824] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:35.549939 25226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc: No bootstrap required, opened a new log
I20260925 09:25:35.554636 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:35.529945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44747 (local address 127.3.82.62:43849)
0925 09:25:35.530853 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:35.530866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:35.530912 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:25:35.533062 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:35.533087 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:35.533552 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:35.534027 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:35.535757 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:35.537165 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:35.539180 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:35.539960 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:35.540097 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:35.545598 (+ 5501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:35.545613 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:35.545651 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:35.545705 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:35.549506 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:35.550639 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:35.550646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:35.550657 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:35.550855 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:35.551444 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:35.551448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:35.551451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:35.552041 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:35.552509 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:35.553032 (+ 523us) server_negotiation.cc:299] Negotiation successful
0925 09:25:35.553391 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":425,"threads_started":1}
I20260925 09:25:35.559098 25226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43849 } }
I20260925 09:25:35.559558 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:35.559576 25226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:35.559680 25226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3dfcb185bc46278ec09c60e158b9cc, State: Initialized, Role: FOLLOWER
I20260925 09:25:35.560290 25226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43849 } }
I20260925 09:25:35.560541 25226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:35.560678 25226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:35.560868 25226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:35.563442 25226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43849 } }
I20260925 09:25:35.564071 25226 leader_election.cc:302] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b3dfcb185bc46278ec09c60e158b9cc; no voters:
I20260925 09:25:35.564790 25226 leader_election.cc:288] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:35.564906 25235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:35.565647 25235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [term 1 LEADER]: Becoming Leader. State: Replica: 4b3dfcb185bc46278ec09c60e158b9cc, State: Running, Role: LEADER
I20260925 09:25:35.566489 25235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43849 } }
I20260925 09:25:35.569108 25237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b3dfcb185bc46278ec09c60e158b9cc. Latest consensus state: current_term: 1 leader_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43849 } } }
I20260925 09:25:35.569567 25237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:35.570254 25238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3dfcb185bc46278ec09c60e158b9cc" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43849 } } }
I20260925 09:25:35.570694 25238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:35.570824 25226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:35.573496 25240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:35.589692 25240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:35.598063 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:35.564932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32907 (local address 127.3.82.62:43849)
0925 09:25:35.565216 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:35.565225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:35.573217 (+ 7992us) server_negotiation.cc:407] Connection header received
0925 09:25:35.573492 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:35.573502 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:35.573610 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:35.573740 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:35.576923 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:35.578574 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:35.581313 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:35.582409 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:35.583011 (+ 602us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:35.587214 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:35.587225 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:35.587233 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:35.587295 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:35.592396 (+ 5101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:35.593912 (+ 1516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:35.593921 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:35.593927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:35.594130 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:35.594899 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:35.594907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:35.594913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:35.595395 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:35.597154 (+ 1759us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:35.597343 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 09:25:35.597599 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 09:25:35.602669 25240 catalog_manager.cc:1347] Generated new cluster ID: c1cdc0bb26e04b138edf6b6c3f95c147
I20260925 09:25:35.602931 25240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:35.620574 25240 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:35.621995 25240 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:35.632826 25240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b3dfcb185bc46278ec09c60e158b9cc: Generated new TSK 0
I20260925 09:25:35.634084 25240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:35.641125 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25159
I20260925 09:25:35.642431 25221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:35.737053 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25159
W20260925 09:25:35.755184 25144 connection.cc:535] client connection to 127.3.82.62:43849 recv error: Network error: failed to read from TLS socket (remote: 127.3.82.62:43849): Connection reset by peer (error 104)
2026-09-25T09:25:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2155 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:35 dist-test-slave-4wm5 krb5kdc[25258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:35 dist-test-slave-4wm5 krb5kdc[25258](info): set up 2 sockets
Sep 25 09:25:35 dist-test-slave-4wm5 krb5kdc[25258](info): commencing operation
krb5kdc: starting...
W20260925 09:25:36.886499 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:37 dist-test-slave-4wm5 krb5kdc[25258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:37.178419 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:46797
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:46187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:46797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:37.526355 25271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:37.526710 25271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:37.526872 25271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:37.534641 25271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:37.534762 25271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:37.534870 25271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:37.534981 25271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:37.535068 25271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:37.546012 25271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:46187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:46797
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:46797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:37.547427 25271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:37.548846 25271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:37.561789 25276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:37.561857 25279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:37.561770 25277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:37.562839 25271 server_base.cc:1034] running on GCE node
I20260925 09:25:37.563768 25271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:37.565696 25271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:37.567008 25271 hybrid_clock.cc:648] HybridClock initialized: now 1790328337566941 us; error 65 us; skew 500 ppm
I20260925 09:25:37.567426 25271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:37 dist-test-slave-4wm5 krb5kdc[25258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328337, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:37.571282 25271 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:37.573123 25271 webserver.cc:460] Webserver started at http://127.3.82.62:40855/ using document root <none> and password file <none>
I20260925 09:25:37.573948 25271 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:37.574101 25271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:37.574419 25271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:37.577657 25271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "2a60d0a37ddd4001aaa885786438a811"
format_stamp: "Formatted at 2026-09-25 09:25:37 on dist-test-slave-4wm5"
server_key: "4f940a2b1a94050397c87ca23f7973c7"
server_key_iv: "758376902353e2db2ad63ae1d467c9b6"
server_key_version: "encryptionkey@0"
I20260925 09:25:37.578608 25271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "2a60d0a37ddd4001aaa885786438a811"
format_stamp: "Formatted at 2026-09-25 09:25:37 on dist-test-slave-4wm5"
server_key: "4f940a2b1a94050397c87ca23f7973c7"
server_key_iv: "758376902353e2db2ad63ae1d467c9b6"
server_key_version: "encryptionkey@0"
I20260925 09:25:37.585014 25271 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260925 09:25:37.589221 25286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:37.590826 25271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 09:25:37.591063 25271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "2a60d0a37ddd4001aaa885786438a811"
format_stamp: "Formatted at 2026-09-25 09:25:37 on dist-test-slave-4wm5"
server_key: "4f940a2b1a94050397c87ca23f7973c7"
server_key_iv: "758376902353e2db2ad63ae1d467c9b6"
server_key_version: "encryptionkey@0"
I20260925 09:25:37.591306 25271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:37.607568 25271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:37.612090 25271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:37.612411 25271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:37.632085 25271 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:46797
I20260925 09:25:37.632128 25337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:46797 every 8 connection(s)
I20260925 09:25:37.633603 25271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:37.636693 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25271
I20260925 09:25:37.636878 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:37.637427 3400 external_mini_cluster.cc:1428] Setting key 65be200130be2f29bde25688155359ed
I20260925 09:25:37.638202 25338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:37.652215 25338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811: Bootstrap starting.
I20260925 09:25:37.652809 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:37.639403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38593 (local address 127.3.82.62:46797)
0925 09:25:37.640440 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:37.640453 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:37.640498 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:25:37.642374 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:37.642400 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:37.642911 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:37.643392 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:37.644051 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:37.645609 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:37.647281 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:37.648106 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:37.648356 (+ 250us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:37.650066 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:37.650077 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:37.650098 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:37.650774 (+ 676us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:37.650871 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:37.650906 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:37.651098 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:37.651440 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 09:25:37.651640 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":551,"threads_started":1}
I20260925 09:25:37.656106 25338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:37.657382 25338 log.cc:824] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:37.657984 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:37.660557 25338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811: No bootstrap required, opened a new log
I20260925 09:25:37.670980 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:37.662949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54957 (local address 127.3.82.62:46797)
0925 09:25:37.663202 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:37.663212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:37.663663 (+ 451us) server_negotiation.cc:407] Connection header received
0925 09:25:37.663885 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:37.663890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:37.663956 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:37.664053 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:37.664766 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:37.665642 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:37.667652 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:37.668183 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:37.668326 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:37.669720 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:37.669726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:37.669729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:37.669752 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:37.669800 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:37.669817 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:37.670176 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:37.670500 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 09:25:37.670569 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:25:37.671280 25338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a60d0a37ddd4001aaa885786438a811" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 46797 } }
I20260925 09:25:37.671759 25338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:37.671908 25338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a60d0a37ddd4001aaa885786438a811, State: Initialized, Role: FOLLOWER
I20260925 09:25:37.672536 25338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a60d0a37ddd4001aaa885786438a811" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 46797 } }
I20260925 09:25:37.672794 25338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:37.672945 25338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:37.673168 25338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:37.677331 25338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a60d0a37ddd4001aaa885786438a811" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 46797 } }
I20260925 09:25:37.678179 25338 leader_election.cc:302] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a60d0a37ddd4001aaa885786438a811; no voters:
I20260925 09:25:37.678941 25338 leader_election.cc:288] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:37.679133 25348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:37.679940 25348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [term 1 LEADER]: Becoming Leader. State: Replica: 2a60d0a37ddd4001aaa885786438a811, State: Running, Role: LEADER
I20260925 09:25:37.681384 25348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a60d0a37ddd4001aaa885786438a811" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 46797 } }
I20260925 09:25:37.683442 25338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:37.685213 25350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a60d0a37ddd4001aaa885786438a811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a60d0a37ddd4001aaa885786438a811" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 46797 } } }
I20260925 09:25:37.685719 25350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:37.686655 25349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a60d0a37ddd4001aaa885786438a811. Latest consensus state: current_term: 1 leader_uuid: "2a60d0a37ddd4001aaa885786438a811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a60d0a37ddd4001aaa885786438a811" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 46797 } } }
I20260925 09:25:37.688530 25349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:37.689776 25353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:37.696691 25353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:37.706853 25353 catalog_manager.cc:1347] Generated new cluster ID: 118768008285497b820da8bcb72f88e1
I20260925 09:25:37.707001 25353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:37.731581 25353 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:37.734035 25353 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:37.751515 25353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a60d0a37ddd4001aaa885786438a811: Generated new TSK 0
I20260925 09:25:37.752482 25353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:37.770778 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25271
W20260925 09:25:37.772142 25333 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38593
W20260925 09:25:37.773102 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25271: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:37.773319 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25271
W20260925 09:25:37.791541 25256 connection.cc:535] client connection to 127.3.82.62:46797 recv error: Network error: failed to read from TLS socket (remote: 127.3.82.62:46797): Connection reset by peer (error 104)
2026-09-25T09:25:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:37 dist-test-slave-4wm5 krb5kdc[25369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:37 dist-test-slave-4wm5 krb5kdc[25369](info): set up 2 sockets
Sep 25 09:25:37 dist-test-slave-4wm5 krb5kdc[25369](info): commencing operation
krb5kdc: starting...
W20260925 09:25:38.906524 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:39 dist-test-slave-4wm5 krb5kdc[25369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:39.199395 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:40585
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:46229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:40585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:39.559967 25382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:39.560284 25382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:39.560384 25382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:39.568140 25382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:39.568277 25382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:39.568324 25382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:39.568377 25382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:39.568423 25382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:39.579027 25382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:46229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:40585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:40585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:39.580363 25382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:39.581712 25382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:39.594635 25387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:39.594594 25388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:39.594676 25390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:39.595407 25382 server_base.cc:1034] running on GCE node
I20260925 09:25:39.596321 25382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:39.597929 25382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:39.599215 25382 hybrid_clock.cc:648] HybridClock initialized: now 1790328339599149 us; error 50 us; skew 500 ppm
I20260925 09:25:39.599648 25382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:39 dist-test-slave-4wm5 krb5kdc[25369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:39.603238 25382 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:39.604986 25382 webserver.cc:460] Webserver started at http://127.3.82.62:39409/ using document root <none> and password file <none>
I20260925 09:25:39.605762 25382 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:39.605942 25382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:39.606313 25382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:39.609706 25382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "4e332064d66348b882747e7b170046ed"
format_stamp: "Formatted at 2026-09-25 09:25:39 on dist-test-slave-4wm5"
server_key: "555e775a53e799cf6c6762249e9c965c"
server_key_iv: "e6ff3ce28d14f1428a77c081bd4050b9"
server_key_version: "encryptionkey@0"
I20260925 09:25:39.610754 25382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "4e332064d66348b882747e7b170046ed"
format_stamp: "Formatted at 2026-09-25 09:25:39 on dist-test-slave-4wm5"
server_key: "555e775a53e799cf6c6762249e9c965c"
server_key_iv: "e6ff3ce28d14f1428a77c081bd4050b9"
server_key_version: "encryptionkey@0"
I20260925 09:25:39.617457 25382 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:25:39.621560 25397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:39.623193 25382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:25:39.623414 25382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "4e332064d66348b882747e7b170046ed"
format_stamp: "Formatted at 2026-09-25 09:25:39 on dist-test-slave-4wm5"
server_key: "555e775a53e799cf6c6762249e9c965c"
server_key_iv: "e6ff3ce28d14f1428a77c081bd4050b9"
server_key_version: "encryptionkey@0"
I20260925 09:25:39.623656 25382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:39.630538 25382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:39.635422 25382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:39.635782 25382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:39.657917 25382 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:40585
I20260925 09:25:39.658071 25448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:40585 every 8 connection(s)
I20260925 09:25:39.659601 25382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:39.663684 25449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:39.667798 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25382
I20260925 09:25:39.667981 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:39.668535 3400 external_mini_cluster.cc:1428] Setting key 7f745d7079cdb3e5464d480eb4b6bc76
I20260925 09:25:39.677977 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:39.670802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40599 (local address 127.3.82.62:40585)
0925 09:25:39.671941 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:39.671954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:39.672003 (+ 49us) server_negotiation.cc:407] Connection header received
0925 09:25:39.673985 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:39.674012 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:39.674503 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:39.674905 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:39.676104 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:39.676111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:39.676126 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:39.676142 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:39.676208 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:39.676229 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:39.676421 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:39.676532 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 09:25:39.676767 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":534,"threads_started":1}
I20260925 09:25:39.678473 25449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed: Bootstrap starting.
I20260925 09:25:39.682286 25449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:39.682483 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:39.683416 25449 log.cc:824] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:39.686811 25449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed: No bootstrap required, opened a new log
I20260925 09:25:39.692750 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:39.687574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49765 (local address 127.3.82.62:40585)
0925 09:25:39.687764 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:39.687771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:39.690495 (+ 2724us) server_negotiation.cc:407] Connection header received
0925 09:25:39.690634 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:39.690639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:39.690705 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:39.690818 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:39.692010 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:39.692016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:39.692020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:39.692044 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:39.692094 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:39.692116 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:39.692250 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:39.692357 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 09:25:39.692431 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 09:25:39.695736 25449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e332064d66348b882747e7b170046ed" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40585 } }
I20260925 09:25:39.696159 25449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:39.696285 25449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e332064d66348b882747e7b170046ed, State: Initialized, Role: FOLLOWER
I20260925 09:25:39.696870 25449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e332064d66348b882747e7b170046ed" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40585 } }
I20260925 09:25:39.697139 25449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:39.697294 25449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:39.697474 25449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:39.699798 25449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e332064d66348b882747e7b170046ed" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40585 } }
I20260925 09:25:39.700345 25449 leader_election.cc:302] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e332064d66348b882747e7b170046ed; no voters:
I20260925 09:25:39.701062 25449 leader_election.cc:288] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:39.701165 25459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:39.701864 25459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [term 1 LEADER]: Becoming Leader. State: Replica: 4e332064d66348b882747e7b170046ed, State: Running, Role: LEADER
I20260925 09:25:39.702566 25459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e332064d66348b882747e7b170046ed" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40585 } }
I20260925 09:25:39.703537 25449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:39.705531 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e332064d66348b882747e7b170046ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e332064d66348b882747e7b170046ed" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40585 } } }
I20260925 09:25:39.705997 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:39.706775 25461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e332064d66348b882747e7b170046ed. Latest consensus state: current_term: 1 leader_uuid: "4e332064d66348b882747e7b170046ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e332064d66348b882747e7b170046ed" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 40585 } } }
I20260925 09:25:39.708005 25461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:39.710798 25465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:39.716275 25465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:39.725279 25465 catalog_manager.cc:1347] Generated new cluster ID: 2284c84410f349668f37c24384a748fa
I20260925 09:25:39.725421 25465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:39.741499 25465 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:39.742054 25465 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:39.750384 25465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e332064d66348b882747e7b170046ed: Generated new TSK 0
I20260925 09:25:39.751297 25465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:39.760408 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25382
W20260925 09:25:39.762457 25444 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40599
W20260925 09:25:39.763304 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25382: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:39.763485 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25382
2026-09-25T09:25:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1990 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:39 dist-test-slave-4wm5 krb5kdc[25480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:39 dist-test-slave-4wm5 krb5kdc[25480](info): set up 2 sockets
Sep 25 09:25:39 dist-test-slave-4wm5 krb5kdc[25480](info): commencing operation
krb5kdc: starting...
W20260925 09:25:40.890520 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:41 dist-test-slave-4wm5 krb5kdc[25480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:41.174817 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42127
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:42127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:41.531390 25493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:41.531769 25493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:41.531955 25493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:41.539641 25493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:41.539850 25493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:41.539974 25493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:41.540088 25493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:41.540181 25493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:41.550787 25493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:42127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:42127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:41.552223 25493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:41.553572 25493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:41.566473 25498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:41.566411 25499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:41.566499 25501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:41.567726 25493 server_base.cc:1034] running on GCE node
I20260925 09:25:41.568686 25493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:41.570334 25493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:41.571640 25493 hybrid_clock.cc:648] HybridClock initialized: now 1790328341571583 us; error 51 us; skew 500 ppm
I20260925 09:25:41.572088 25493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:41 dist-test-slave-4wm5 krb5kdc[25480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328341, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:41.581415 25493 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:41.583597 25493 webserver.cc:460] Webserver started at http://127.3.82.62:34751/ using document root <none> and password file <none>
I20260925 09:25:41.584419 25493 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:41.584586 25493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:41.584944 25493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:41.588282 25493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "bd538ec74a6d422ab50924e4a0db0857"
format_stamp: "Formatted at 2026-09-25 09:25:41 on dist-test-slave-4wm5"
server_key: "0b68d0b141eedd4bec828dd0894f5eda"
server_key_iv: "30f6cda210188062d8f40312bc21cb50"
server_key_version: "encryptionkey@0"
I20260925 09:25:41.589145 25493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "bd538ec74a6d422ab50924e4a0db0857"
format_stamp: "Formatted at 2026-09-25 09:25:41 on dist-test-slave-4wm5"
server_key: "0b68d0b141eedd4bec828dd0894f5eda"
server_key_iv: "30f6cda210188062d8f40312bc21cb50"
server_key_version: "encryptionkey@0"
I20260925 09:25:41.594878 25493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 09:25:41.598809 25508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:41.600356 25493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 09:25:41.600566 25493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "bd538ec74a6d422ab50924e4a0db0857"
format_stamp: "Formatted at 2026-09-25 09:25:41 on dist-test-slave-4wm5"
server_key: "0b68d0b141eedd4bec828dd0894f5eda"
server_key_iv: "30f6cda210188062d8f40312bc21cb50"
server_key_version: "encryptionkey@0"
I20260925 09:25:41.600788 25493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:41.634675 25493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:41.639803 25493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:41.640134 25493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:41.660483 25493 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:42127
I20260925 09:25:41.660521 25559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:42127 every 8 connection(s)
I20260925 09:25:41.662091 25493 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:41.663795 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25493
I20260925 09:25:41.663983 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:41.664422 3400 external_mini_cluster.cc:1428] Setting key 2142fa9b6bc4f761c6a8a7faa36574f0
I20260925 09:25:41.667100 25560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 09:25:41 dist-test-slave-4wm5 krb5kdc[25480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:41.682988 25560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857: Bootstrap starting.
I20260925 09:25:41.687062 25560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:41.688475 25560 log.cc:824] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:41.690887 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:41.666393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35673 (local address 127.3.82.62:42127)
0925 09:25:41.667287 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:41.667301 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:41.667343 (+ 42us) server_negotiation.cc:407] Connection header received
0925 09:25:41.669100 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:41.669123 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:41.669565 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:41.670055 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:41.672083 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:41.673570 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:41.675695 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:41.676595 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:41.676744 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:41.682301 (+ 5557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:41.682311 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:41.682332 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:41.682385 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:41.685750 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:41.686662 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:41.686671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:41.686678 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:41.686858 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:41.687329 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:41.687333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:41.687335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:41.687966 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:41.688212 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:41.688939 (+ 727us) server_negotiation.cc:299] Negotiation successful
0925 09:25:41.689500 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":403,"threads_started":1}
I20260925 09:25:41.692282 25560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857: No bootstrap required, opened a new log
I20260925 09:25:41.695348 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:41.700958 25560 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd538ec74a6d422ab50924e4a0db0857" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42127 } }
I20260925 09:25:41.701506 25560 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:41.701630 25560 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd538ec74a6d422ab50924e4a0db0857, State: Initialized, Role: FOLLOWER
I20260925 09:25:41.702454 25560 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd538ec74a6d422ab50924e4a0db0857" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42127 } }
I20260925 09:25:41.702751 25560 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:41.702906 25560 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:41.703125 25560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:41.706326 25560 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd538ec74a6d422ab50924e4a0db0857" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42127 } }
I20260925 09:25:41.706992 25560 leader_election.cc:302] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd538ec74a6d422ab50924e4a0db0857; no voters:
I20260925 09:25:41.707774 25560 leader_election.cc:288] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:41.707921 25570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:41.710327 25560 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:41.711185 25570 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [term 1 LEADER]: Becoming Leader. State: Replica: bd538ec74a6d422ab50924e4a0db0857, State: Running, Role: LEADER
I20260925 09:25:41.712026 25570 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd538ec74a6d422ab50924e4a0db0857" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42127 } }
I20260925 09:25:41.716138 25572 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd538ec74a6d422ab50924e4a0db0857. Latest consensus state: current_term: 1 leader_uuid: "bd538ec74a6d422ab50924e4a0db0857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd538ec74a6d422ab50924e4a0db0857" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42127 } } }
I20260925 09:25:41.716279 25571 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd538ec74a6d422ab50924e4a0db0857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd538ec74a6d422ab50924e4a0db0857" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 42127 } } }
I20260925 09:25:41.716621 25572 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:41.716687 25571 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:41.717862 25579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:41.719633 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:41.700166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36031 (local address 127.3.82.62:42127)
0925 09:25:41.700444 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:41.700461 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:41.700790 (+ 329us) server_negotiation.cc:407] Connection header received
0925 09:25:41.700986 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:41.700991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:41.701057 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:41.701146 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:41.703252 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:41.704077 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:41.706003 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:41.706631 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:41.706843 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:41.710078 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:41.710085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:41.710089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:41.710137 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:41.714457 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:41.715734 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:41.715743 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:41.715748 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:41.716014 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:41.716572 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:41.716578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:41.716581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:41.717448 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:41.717776 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:41.718502 (+ 726us) server_negotiation.cc:299] Negotiation successful
0925 09:25:41.718870 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260925 09:25:41.725344 25579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:41.737200 25579 catalog_manager.cc:1347] Generated new cluster ID: 50a7c3d4334b4fd4ae4e53b09f71e73e
I20260925 09:25:41.737360 25579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:41.749495 25579 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:41.751111 25579 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:41.767508 25579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd538ec74a6d422ab50924e4a0db0857: Generated new TSK 0
I20260925 09:25:41.768900 25579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:41.782290 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25493
I20260925 09:25:41.783900 25555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:41.881399 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25493
2026-09-25T09:25:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2117 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:41 dist-test-slave-4wm5 krb5kdc[25592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:41 dist-test-slave-4wm5 krb5kdc[25592](info): set up 2 sockets
Sep 25 09:25:41 dist-test-slave-4wm5 krb5kdc[25592](info): commencing operation
krb5kdc: starting...
W20260925 09:25:43.031112 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:43 dist-test-slave-4wm5 krb5kdc[25592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:43.327446 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43901
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:33881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:43.686971 25605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:43.687297 25605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:43.687398 25605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:43.695078 25605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:43.695212 25605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:43.695289 25605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:43.695358 25605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:43.695405 25605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:43.705991 25605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:33881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:43.707337 25605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:43.708894 25605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:43.722654 25610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:43.722572 25613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:43.722666 25611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:43.723866 25605 server_base.cc:1034] running on GCE node
I20260925 09:25:43.724864 25605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:43.726744 25605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:43.728039 25605 hybrid_clock.cc:648] HybridClock initialized: now 1790328343727991 us; error 43 us; skew 500 ppm
I20260925 09:25:43.728466 25605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:43 dist-test-slave-4wm5 krb5kdc[25592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328343, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:43.732422 25605 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:43.734306 25605 webserver.cc:460] Webserver started at http://127.3.82.62:39959/ using document root <none> and password file <none>
I20260925 09:25:43.735116 25605 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:43.735280 25605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:43.735666 25605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:43.739127 25605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "5eee7fc702914deb9e426355babeaa16"
format_stamp: "Formatted at 2026-09-25 09:25:43 on dist-test-slave-4wm5"
server_key: "85594de203034267fe471c018d55f7a2"
server_key_iv: "574adab86f30d30bb2f65eb55f09f161"
server_key_version: "encryptionkey@0"
I20260925 09:25:43.740028 25605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "5eee7fc702914deb9e426355babeaa16"
format_stamp: "Formatted at 2026-09-25 09:25:43 on dist-test-slave-4wm5"
server_key: "85594de203034267fe471c018d55f7a2"
server_key_iv: "574adab86f30d30bb2f65eb55f09f161"
server_key_version: "encryptionkey@0"
I20260925 09:25:43.746416 25605 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:25:43.750838 25620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:43.752779 25605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 09:25:43.753136 25605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "5eee7fc702914deb9e426355babeaa16"
format_stamp: "Formatted at 2026-09-25 09:25:43 on dist-test-slave-4wm5"
server_key: "85594de203034267fe471c018d55f7a2"
server_key_iv: "574adab86f30d30bb2f65eb55f09f161"
server_key_version: "encryptionkey@0"
I20260925 09:25:43.753391 25605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:43.769264 25605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:43.774766 25605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:43.775115 25605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:43.796986 25605 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:43901
I20260925 09:25:43.796984 25671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:43901 every 8 connection(s)
I20260925 09:25:43.798671 25605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:43.803025 25672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:43.807178 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25605
I20260925 09:25:43.807395 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:43.808038 3400 external_mini_cluster.cc:1428] Setting key af7367c82929684dd46d362ba77fdd88
I20260925 09:25:43.817507 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16: Bootstrap starting.
I20260925 09:25:43.821815 25672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:43.823120 25672 log.cc:824] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:43.823774 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:43.810276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47105 (local address 127.3.82.62:43901)
0925 09:25:43.811142 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:43.811154 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:43.811198 (+ 44us) server_negotiation.cc:407] Connection header received
0925 09:25:43.812962 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:43.812991 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:43.813517 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:43.814013 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:43.814694 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:43.816222 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:43.818263 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:43.819209 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:43.819387 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:43.820931 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:43.820942 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:43.820959 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:43.821557 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:43.821645 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:43.821676 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:43.821774 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:43.822225 (+ 451us) server_negotiation.cc:299] Negotiation successful
0925 09:25:43.822555 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":373,"threads_started":1}
I20260925 09:25:43.826725 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16: No bootstrap required, opened a new log
I20260925 09:25:43.828685 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:43.835440 25672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee7fc702914deb9e426355babeaa16" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43901 } }
I20260925 09:25:43.835932 25672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:43.836072 25672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eee7fc702914deb9e426355babeaa16, State: Initialized, Role: FOLLOWER
I20260925 09:25:43.836699 25672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee7fc702914deb9e426355babeaa16" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43901 } }
I20260925 09:25:43.836987 25672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:43.837146 25672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:43.837342 25672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:43.840137 25672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee7fc702914deb9e426355babeaa16" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43901 } }
I20260925 09:25:43.840754 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:43.833482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55329 (local address 127.3.82.62:43901)
0925 09:25:43.833701 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:43.833710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:43.834209 (+ 499us) server_negotiation.cc:407] Connection header received
0925 09:25:43.834471 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:43.834475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:43.834552 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:43.834667 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:43.835240 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:43.836092 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:43.837811 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:43.838358 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:43.838532 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:43.839920 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:43.839927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:43.839931 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:43.839954 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:43.840005 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:43.840025 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:43.840175 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:43.840316 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 09:25:43.840423 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 09:25:43.840957 25672 leader_election.cc:302] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eee7fc702914deb9e426355babeaa16; no voters:
I20260925 09:25:43.841770 25672 leader_election.cc:288] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:43.841921 25682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:43.848917 25682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [term 1 LEADER]: Becoming Leader. State: Replica: 5eee7fc702914deb9e426355babeaa16, State: Running, Role: LEADER
I20260925 09:25:43.849721 25682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee7fc702914deb9e426355babeaa16" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43901 } }
I20260925 09:25:43.850466 25672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:43.852967 25683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eee7fc702914deb9e426355babeaa16. Latest consensus state: current_term: 1 leader_uuid: "5eee7fc702914deb9e426355babeaa16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee7fc702914deb9e426355babeaa16" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43901 } } }
I20260925 09:25:43.853595 25683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:43.853480 25684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eee7fc702914deb9e426355babeaa16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee7fc702914deb9e426355babeaa16" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43901 } } }
I20260925 09:25:43.853940 25684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:43.858891 25691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:43.864382 25691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:43.875471 25691 catalog_manager.cc:1347] Generated new cluster ID: 62549edea91442409d8ba40d1233fee4
I20260925 09:25:43.875661 25691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:43.895083 25691 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:43.896685 25691 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:43.904592 25691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eee7fc702914deb9e426355babeaa16: Generated new TSK 0
I20260925 09:25:43.905658 25691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:43.918426 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25605
W20260925 09:25:43.920006 25667 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47105
W20260925 09:25:43.920786 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25605: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:43.920990 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25605
2026-09-25T09:25:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2038 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:44 dist-test-slave-4wm5 krb5kdc[25703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:44 dist-test-slave-4wm5 krb5kdc[25703](info): set up 2 sockets
Sep 25 09:25:44 dist-test-slave-4wm5 krb5kdc[25703](info): commencing operation
krb5kdc: starting...
W20260925 09:25:45.063033 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:45 dist-test-slave-4wm5 krb5kdc[25703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:45.370885 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37353
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:37353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:45.728670 25716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:45.728994 25716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:45.729089 25716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:45.736599 25716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:45.736739 25716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:45.736786 25716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:45.736847 25716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:45.736893 25716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:45.747310 25716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:37353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:37353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:45.748802 25716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:45.750171 25716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:45.764056 25722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:45.764137 25721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:45.764137 25724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:45.765309 25716 server_base.cc:1034] running on GCE node
I20260925 09:25:45.766288 25716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:45.767897 25716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:45.769287 25716 hybrid_clock.cc:648] HybridClock initialized: now 1790328345769220 us; error 53 us; skew 500 ppm
I20260925 09:25:45.769716 25716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:45 dist-test-slave-4wm5 krb5kdc[25703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328345, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:45.773386 25716 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:45.775272 25716 webserver.cc:460] Webserver started at http://127.3.82.62:36805/ using document root <none> and password file <none>
I20260925 09:25:45.776058 25716 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:45.776222 25716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:45.776571 25716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:45.779994 25716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "dc9574c48f784fca9d0065932ea0de63"
format_stamp: "Formatted at 2026-09-25 09:25:45 on dist-test-slave-4wm5"
server_key: "269a9bfc8371b97b6b25fe3a7e2f24f0"
server_key_iv: "721c25824b3cf894db8d55c4452331c9"
server_key_version: "encryptionkey@0"
I20260925 09:25:45.780794 25716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "dc9574c48f784fca9d0065932ea0de63"
format_stamp: "Formatted at 2026-09-25 09:25:45 on dist-test-slave-4wm5"
server_key: "269a9bfc8371b97b6b25fe3a7e2f24f0"
server_key_iv: "721c25824b3cf894db8d55c4452331c9"
server_key_version: "encryptionkey@0"
I20260925 09:25:45.786566 25716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 09:25:45.790975 25731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:45.792737 25716 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:25:45.793006 25716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "dc9574c48f784fca9d0065932ea0de63"
format_stamp: "Formatted at 2026-09-25 09:25:45 on dist-test-slave-4wm5"
server_key: "269a9bfc8371b97b6b25fe3a7e2f24f0"
server_key_iv: "721c25824b3cf894db8d55c4452331c9"
server_key_version: "encryptionkey@0"
I20260925 09:25:45.793272 25716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:45.800462 25716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:45.805269 25716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:45.805604 25716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:45.826437 25716 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:37353
I20260925 09:25:45.826534 25782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:37353 every 8 connection(s)
I20260925 09:25:45.828029 25716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:45.829754 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25716
I20260925 09:25:45.830087 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:45.830667 3400 external_mini_cluster.cc:1428] Setting key 0cb0b1d6a95b9351410fd41054050eda
I20260925 09:25:45.833235 25783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:45.840063 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:45.832798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58093 (local address 127.3.82.62:37353)
0925 09:25:45.833796 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:45.833810 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:45.833925 (+ 115us) server_negotiation.cc:407] Connection header received
0925 09:25:45.836006 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:45.836047 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:45.836579 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:45.837067 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:45.838251 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:45.838259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:45.838276 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:45.838292 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:45.838365 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:45.838389 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:45.838679 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:45.838836 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 09:25:45.839050 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":438,"threads_started":1}
I20260925 09:25:45.844691 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:45.847002 25783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63: Bootstrap starting.
I20260925 09:25:45.851037 25783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:45.852533 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:45.849381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33123 (local address 127.3.82.62:37353)
0925 09:25:45.849591 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:45.849599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:45.849999 (+ 400us) server_negotiation.cc:407] Connection header received
0925 09:25:45.850167 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:45.850173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:45.850241 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:45.850368 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:45.851675 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:45.851681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:45.851685 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 09:25:45.851705 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:45.851758 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 09:25:45.851777 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:45.851904 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:45.852126 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 09:25:45.852190 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260925 09:25:45.852605 25783 log.cc:824] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:45.856065 25783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63: No bootstrap required, opened a new log
I20260925 09:25:45.865154 25783 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9574c48f784fca9d0065932ea0de63" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37353 } }
I20260925 09:25:45.865564 25783 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:25:45.865700 25783 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9574c48f784fca9d0065932ea0de63, State: Initialized, Role: FOLLOWER
I20260925 09:25:45.866396 25783 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9574c48f784fca9d0065932ea0de63" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37353 } }
I20260925 09:25:45.866632 25783 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 09:25:45.866801 25783 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 09:25:45.866978 25783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:45.869259 25783 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9574c48f784fca9d0065932ea0de63" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37353 } }
I20260925 09:25:45.869817 25783 leader_election.cc:302] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9574c48f784fca9d0065932ea0de63; no voters:
I20260925 09:25:45.870533 25783 leader_election.cc:288] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 09:25:45.870713 25793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:45.871443 25793 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [term 1 LEADER]: Becoming Leader. State: Replica: dc9574c48f784fca9d0065932ea0de63, State: Running, Role: LEADER
I20260925 09:25:45.872161 25793 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9574c48f784fca9d0065932ea0de63" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37353 } }
I20260925 09:25:45.873036 25783 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:45.875543 25795 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc9574c48f784fca9d0065932ea0de63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9574c48f784fca9d0065932ea0de63" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37353 } } }
I20260925 09:25:45.875560 25794 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9574c48f784fca9d0065932ea0de63. Latest consensus state: current_term: 1 leader_uuid: "dc9574c48f784fca9d0065932ea0de63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9574c48f784fca9d0065932ea0de63" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 37353 } } }
I20260925 09:25:45.876032 25794 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:45.876032 25795 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:45.878428 25801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:45.884375 25801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:45.895452 25801 catalog_manager.cc:1347] Generated new cluster ID: 951505f558a0430b882ccd23eb8ffd09
I20260925 09:25:45.895637 25801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:45.923285 25801 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:45.923804 25801 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:45.932269 25801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc9574c48f784fca9d0065932ea0de63: Generated new TSK 0
I20260925 09:25:45.933554 25801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:45.951231 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25716
W20260925 09:25:45.952437 25778 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58093
W20260925 09:25:45.953190 3400 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25716: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 09:25:45.953359 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25716
2026-09-25T09:25:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2033 ms)
[----------] 18 tests from AuthTokenIssuingTest (37237 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:46 dist-test-slave-4wm5 krb5kdc[25814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:46 dist-test-slave-4wm5 krb5kdc[25814](info): set up 2 sockets
Sep 25 09:25:46 dist-test-slave-4wm5 krb5kdc[25814](info): commencing operation
krb5kdc: starting...
W20260925 09:25:47.102694 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:47 dist-test-slave-4wm5 krb5kdc[25814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:47.394672 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43143
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43143,127.3.82.61:42037,127.3.82.60:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:47.758394 25827 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:25:47.758847 25827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:47.758994 25827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:47.759140 25827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:47.767104 25827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:47.767262 25827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:47.767373 25827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:47.767468 25827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:47.767614 25827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:47.778388 25827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43143,127.3.82.61:42037,127.3.82.60:43947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:43143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:47.779860 25827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:47.781581 25827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:47.795358 25832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:47.795450 25835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:47.795411 25833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:47.796648 25827 server_base.cc:1034] running on GCE node
I20260925 09:25:47.797613 25827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:47.799254 25827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:47.800510 25827 hybrid_clock.cc:648] HybridClock initialized: now 1790328347800431 us; error 38 us; skew 500 ppm
I20260925 09:25:47.801008 25827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:47 dist-test-slave-4wm5 krb5kdc[25814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:47.809661 25827 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:47.811406 25827 webserver.cc:460] Webserver started at http://127.3.82.62:39485/ using document root <none> and password file <none>
I20260925 09:25:47.812158 25827 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:47.812258 25827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:47.812523 25827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:47.815780 25827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "068106a6633c4553954b4c813e88384e"
format_stamp: "Formatted at 2026-09-25 09:25:47 on dist-test-slave-4wm5"
server_key: "1f39db6ccdd80689a6762bb576cd37f1"
server_key_iv: "b09216bd43d236e784011f710628f0f2"
server_key_version: "encryptionkey@0"
I20260925 09:25:47.816691 25827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "068106a6633c4553954b4c813e88384e"
format_stamp: "Formatted at 2026-09-25 09:25:47 on dist-test-slave-4wm5"
server_key: "1f39db6ccdd80689a6762bb576cd37f1"
server_key_iv: "b09216bd43d236e784011f710628f0f2"
server_key_version: "encryptionkey@0"
I20260925 09:25:47.822736 25827 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 09:25:47.826866 25842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:47.828678 25827 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260925 09:25:47.828959 25827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "068106a6633c4553954b4c813e88384e"
format_stamp: "Formatted at 2026-09-25 09:25:47 on dist-test-slave-4wm5"
server_key: "1f39db6ccdd80689a6762bb576cd37f1"
server_key_iv: "b09216bd43d236e784011f710628f0f2"
server_key_version: "encryptionkey@0"
I20260925 09:25:47.829262 25827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:47.841411 25827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:47.846623 25827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:47.846928 25827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:47.868122 25827 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:43143
I20260925 09:25:47.868122 25893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:43143 every 8 connection(s)
I20260925 09:25:47.869796 25827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:47.874222 25894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:47.874358 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25827
I20260925 09:25:47.874578 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:47.875366 3400 external_mini_cluster.cc:1428] Setting key 3513f146e7f22ca38c5c019f5ce71ddb
I20260925 09:25:47.885061 25894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:47 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.61@KRBTEST.COM" created.
I20260925 09:25:47.915091 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:47.889094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43143 (local address 127.0.0.1:37708)
0925 09:25:47.890167 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:47.890225 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:47.893293 (+ 3068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:47.894553 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:47.894568 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:47.895455 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:47.896025 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:47.896057 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:47.898131 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:47.898139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:47.900177 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:47.900197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:47.901373 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:47.901382 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:47.901635 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:47.902498 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:47.902541 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:47.905635 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:47.910300 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:47.910318 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:47.910336 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:47.911054 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:47.911553 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:47.911561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:47.911567 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:47.911797 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:47.912729 (+ 932us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:47.912741 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:47.913055 (+ 314us) client_negotiation.cc:769] Sending connection context
0925 09:25:47.913346 (+ 291us) client_negotiation.cc:240] Negotiation successful
0925 09:25:47.914054 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":134,"thread_start_us":412,"threads_started":1}
I20260925 09:25:47.915107 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:47.889094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.3.82.62:43143)
0925 09:25:47.891387 (+ 2293us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:47.891399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:47.891445 (+ 46us) server_negotiation.cc:407] Connection header received
0925 09:25:47.893924 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:47.893932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:47.894234 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:47.894478 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:47.896291 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:47.897928 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:47.900377 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:47.901067 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:47.901247 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:47.905988 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:47.906005 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:47.906025 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:47.906078 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:47.910121 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:47.911237 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:47.911245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:47.911248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:47.911412 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:47.911916 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:47.911920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:47.911922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:47.912556 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:47.912718 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:47.913476 (+ 758us) server_negotiation.cc:299] Negotiation successful
0925 09:25:47.913903 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":2042,"spinlock_wait_cycles":25344,"thread_start_us":1779,"threads_started":1}
I20260925 09:25:47.918498 25894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:47.919839 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:47.919306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58198)
0925 09:25:47.919507 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:47.919667 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260925 09:25:47.920224 25845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
I20260925 09:25:47.922034 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:47.921483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58202)
0925 09:25:47.921680 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:47.921792 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 09:25:47.922267 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:42037: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab.
Entry for principal kudu/127.3.82.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab.
I20260925 09:25:47.946666 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:42037
--webserver_interface=127.3.82.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab
--principal=kudu/127.3.82.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43143,127.3.82.61:42037,127.3.82.60:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:25:47.971659 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } attempt: 1
I20260925 09:25:47.973048 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:47.972497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58216)
0925 09:25:47.972737 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:47.972862 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260925 09:25:47.974256 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:47.973748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58224)
0925 09:25:47.973937 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:47.974053 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 09:25:47.974448 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:42037: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
I20260925 09:25:48.042742 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } attempt: 2
I20260925 09:25:48.044191 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.043606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58240)
0925 09:25:48.043864 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.043987 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 09:25:48.045616 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.045130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58242)
0925 09:25:48.045303 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.045425 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:25:48.045941 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:42037: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
I20260925 09:25:48.137272 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } attempt: 3
I20260925 09:25:48.138772 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.138159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58246)
0925 09:25:48.138417 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.138557 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260925 09:25:48.140314 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.139716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58252)
0925 09:25:48.139943 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.140065 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260925 09:25:48.140692 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:42037: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
I20260925 09:25:48.283973 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } attempt: 4
I20260925 09:25:48.285354 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.284646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58260)
0925 09:25:48.285047 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.285162 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260925 09:25:48.286751 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.286233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58270)
0925 09:25:48.286427 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.286559 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:25:48.287046 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:42037: Network error: Client connection negotiation failed: client connection to 127.3.82.61:42037: connect: Connection refused (error 111)
W20260925 09:25:48.319182 25901 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:25:48.319573 25901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:48.319693 25901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:48.319834 25901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:48.327977 25901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:48.328095 25901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:48.328193 25901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:48.328290 25901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:48.328406 25901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:48.339452 25901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43143,127.3.82.61:42037,127.3.82.60:43947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab
--principal=kudu/127.3.82.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:42037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.82.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:48.340907 25901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:48.342433 25901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:48.356832 25907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:48.356946 25906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:48.356969 25909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:48.358400 25901 server_base.cc:1034] running on GCE node
I20260925 09:25:48.359406 25901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:48.361101 25901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:48.362386 25901 hybrid_clock.cc:648] HybridClock initialized: now 1790328348362298 us; error 41 us; skew 500 ppm
I20260925 09:25:48.362902 25901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:48 dist-test-slave-4wm5 krb5kdc[25814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:48.366923 25901 init.cc:377] Logged in from keytab as kudu/127.3.82.61@KRBTEST.COM (short username kudu)
I20260925 09:25:48.368813 25901 webserver.cc:460] Webserver started at http://127.3.82.61:37511/ using document root <none> and password file <none>
I20260925 09:25:48.369573 25901 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:48.369678 25901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:48.370090 25901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:48.373494 25901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data/instance:
uuid: "4d916084094e4e22b5fc4e47cad671a0"
format_stamp: "Formatted at 2026-09-25 09:25:48 on dist-test-slave-4wm5"
server_key: "5de7cca92dd6af6f9f3772b0cfafc25f"
server_key_iv: "505ae5bbfb4400c4dca5baf2d7f4e8fc"
server_key_version: "encryptionkey@0"
I20260925 09:25:48.374497 25901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal/instance:
uuid: "4d916084094e4e22b5fc4e47cad671a0"
format_stamp: "Formatted at 2026-09-25 09:25:48 on dist-test-slave-4wm5"
server_key: "5de7cca92dd6af6f9f3772b0cfafc25f"
server_key_iv: "505ae5bbfb4400c4dca5baf2d7f4e8fc"
server_key_version: "encryptionkey@0"
I20260925 09:25:48.381008 25901 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260925 09:25:48.386457 25916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:48.388381 25901 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260925 09:25:48.388600 25901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal
uuid: "4d916084094e4e22b5fc4e47cad671a0"
format_stamp: "Formatted at 2026-09-25 09:25:48 on dist-test-slave-4wm5"
server_key: "5de7cca92dd6af6f9f3772b0cfafc25f"
server_key_iv: "505ae5bbfb4400c4dca5baf2d7f4e8fc"
server_key_version: "encryptionkey@0"
I20260925 09:25:48.388851 25901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:48.417253 25901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:48.422286 25901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:48.422614 25901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:48.442801 25901 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.61:42037
I20260925 09:25:48.442847 25967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.61:42037 every 8 connection(s)
I20260925 09:25:48.444271 25901 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
I20260925 09:25:48.446894 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25901
I20260925 09:25:48.447104 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-1/wal/instance
I20260925 09:25:48.447583 3400 external_mini_cluster.cc:1428] Setting key 77cde68307fc8545b51d589ae585e875
I20260925 09:25:48.448624 25968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:48.455667 25968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:48 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.60@KRBTEST.COM" created.
I20260925 09:25:48.479667 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.458657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.3.82.62:43143)
0925 09:25:48.459228 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:48.459236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:48.460291 (+ 1055us) server_negotiation.cc:407] Connection header received
0925 09:25:48.462406 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:48.462415 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:48.462520 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:48.462658 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:48.464319 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.465500 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.467238 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.468028 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.468191 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:48.472231 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:48.472237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:48.472240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:48.472290 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:48.475617 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:48.476776 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:48.476781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:48.476784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:48.477002 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:48.477558 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:48.477562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:48.477564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:48.478027 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:48.478159 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:48.479175 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0925 09:25:48.479382 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":268,"threads_started":1}
I20260925 09:25:48.481215 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.459264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43143 (local address 127.0.0.1:37714)
0925 09:25:48.460100 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.460148 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:48.462100 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:48.462870 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:48.462879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:48.463477 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:48.464073 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:48.464092 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.465680 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.465684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:48.467070 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:48.467083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.468264 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.468269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:48.468493 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:48.469275 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:48.469298 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:48.471958 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:48.475811 (+ 3853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:48.475825 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:48.475847 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:48.476609 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:48.477136 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:48.477144 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:48.477150 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:48.477419 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:48.478170 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:48.478182 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:48.478696 (+ 514us) client_negotiation.cc:769] Sending connection context
0925 09:25:48.479046 (+ 350us) client_negotiation.cc:240] Negotiation successful
0925 09:25:48.479665 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":360,"threads_started":1}
I20260925 09:25:48.483503 25968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:48 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:48.500795 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.484445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58276)
0925 09:25:48.484664 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.484686 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:48.484815 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:48.485623 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:48.485628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:48.486095 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:48.486241 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:48.486249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.487908 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.487917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:48.489476 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:48.489494 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.490612 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.490624 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:48.490828 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:48.491498 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:48.491525 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:48.494525 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:48.498082 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:48.498090 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:48.498096 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:48.498478 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:48.498893 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:48.498898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:48.498902 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:48.499077 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:48.500007 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:48.500011 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:48.500161 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 09:25:48.500285 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 09:25:48.500527 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:25:48.501379 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.484514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58276 (local address 127.3.82.61:42037)
0925 09:25:48.485163 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:48.485169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:48.485197 (+ 28us) server_negotiation.cc:407] Connection header received
0925 09:25:48.485245 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:48.485250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:48.485448 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:48.485608 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:48.486421 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.487734 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.489632 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.490426 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.490531 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:48.494779 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:48.494791 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:48.494807 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:48.494839 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:48.497937 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:48.498619 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:48.498628 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:48.498631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:48.498758 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:48.499234 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:48.499242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:48.499248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:48.499777 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:48.500053 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:48.500505 (+ 452us) server_negotiation.cc:299] Negotiation successful
0925 09:25:48.500845 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":334,"threads_started":1}
I20260925 09:25:48.504020 25968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:48.505221 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.504782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54338)
0925 09:25:48.504943 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.505082 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 09:25:48.505501 25917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.507154 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.506830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54340)
0925 09:25:48.506957 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.507043 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 09:25:48.507369 25968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab.
Entry for principal kudu/127.3.82.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab.
I20260925 09:25:48.514240 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:43947
--webserver_interface=127.3.82.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab
--principal=kudu/127.3.82.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:37451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:43143,127.3.82.61:42037,127.3.82.60:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:25:48.556720 25968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 1
I20260925 09:25:48.557925 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.557423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54356)
0925 09:25:48.557620 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.557716 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:25:48.559119 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.558612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54372)
0925 09:25:48.558848 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.558977 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260925 09:25:48.559365 25968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.586354 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } attempt: 5
Sep 25 09:25:48 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:48.601924 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.587322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.3.82.61:42037)
0925 09:25:48.587609 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:48.587615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:48.587638 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:25:48.587710 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:48.587715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:48.587805 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:48.587933 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:48.588725 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.589805 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.591098 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.591743 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.591883 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:48.595810 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:48.595815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:48.595818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:48.595952 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:48.598867 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:48.599879 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:48.599883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:48.599885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:48.600024 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:48.600461 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:48.600464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:48.600466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:48.600745 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:48.600861 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:48.601400 (+ 539us) server_negotiation.cc:299] Negotiation successful
0925 09:25:48.601615 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260925 09:25:48.601974 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.587217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58292)
0925 09:25:48.587459 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.587489 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:48.587619 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:48.588002 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:48.588006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:48.588376 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:48.588553 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:48.588562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.590044 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.590049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:48.590934 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:48.590946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:48.591927 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:48.591932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:48.592119 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:48.592893 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:48.592919 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:48.595557 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:48.599068 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:48.599076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:48.599081 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:48.599723 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:48.600145 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:48.600150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:48.600153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:48.600344 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:48.600873 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:48.600880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:48.601063 (+ 183us) client_negotiation.cc:769] Sending connection context
0925 09:25:48.601256 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 09:25:48.601565 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260925 09:25:48.603210 25894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:48.604319 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.603829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54378)
0925 09:25:48.604046 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.604143 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 09:25:48.605419 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.605016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54392)
0925 09:25:48.605184 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.605284 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260925 09:25:48.605666 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.627643 25968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 2
I20260925 09:25:48.628988 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.628401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54408)
0925 09:25:48.628647 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.628771 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260925 09:25:48.630148 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.629690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54412)
0925 09:25:48.629896 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.629994 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 09:25:48.630386 25968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.656937 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 1
I20260925 09:25:48.658241 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.657658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54420)
0925 09:25:48.657917 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.658031 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 09:25:48.659411 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.659019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54422)
0925 09:25:48.659180 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.659272 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 09:25:48.659677 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.721665 25968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 3
I20260925 09:25:48.723047 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.722473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54432)
0925 09:25:48.722734 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.722850 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260925 09:25:48.724278 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.723793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54448)
0925 09:25:48.724052 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.724148 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260925 09:25:48.724506 25968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.727923 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 2
I20260925 09:25:48.729377 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.728711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54460)
0925 09:25:48.728984 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.729126 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 09:25:48.730785 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.730250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54466)
0925 09:25:48.730466 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.730581 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260925 09:25:48.731102 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.837384 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 3
I20260925 09:25:48.838716 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.838194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54482)
0925 09:25:48.838408 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.838529 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 09:25:48.840355 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.839659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54488)
0925 09:25:48.840095 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.840201 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274}
W20260925 09:25:48.840588 25894 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
I20260925 09:25:48.867763 25968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 4
I20260925 09:25:48.869100 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.868510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54502)
0925 09:25:48.868790 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.868900 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260925 09:25:48.870321 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:48.869870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54516)
0925 09:25:48.870091 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:48.870177 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260925 09:25:48.870517 25968 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:43947: Network error: Client connection negotiation failed: client connection to 127.3.82.60:43947: connect: Connection refused (error 111)
W20260925 09:25:48.878830 25976 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:25:48.879164 25976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:48.879305 25976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:48.879417 25976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:48.887207 25976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:48.887316 25976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:48.887414 25976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:48.887511 25976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:48.887638 25976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:48.898581 25976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:37451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:43143,127.3.82.61:42037,127.3.82.60:43947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab
--principal=kudu/127.3.82.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:43947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.82.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:48.900435 25976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:48.901894 25976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:48.915230 25982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:48.915230 25981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:48.915266 25984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:48.916288 25976 server_base.cc:1034] running on GCE node
I20260925 09:25:48.917184 25976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:48.918764 25976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:48.920037 25976 hybrid_clock.cc:648] HybridClock initialized: now 1790328348919914 us; error 62 us; skew 500 ppm
I20260925 09:25:48.920444 25976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:48 dist-test-slave-4wm5 krb5kdc[25814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:48.923996 25976 init.cc:377] Logged in from keytab as kudu/127.3.82.60@KRBTEST.COM (short username kudu)
I20260925 09:25:48.925805 25976 webserver.cc:460] Webserver started at http://127.3.82.60:40825/ using document root <none> and password file <none>
I20260925 09:25:48.926625 25976 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:48.926743 25976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:48.927102 25976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:48.930470 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data/instance:
uuid: "f1f75db3b3e343dd83aea3bbc9fabf42"
format_stamp: "Formatted at 2026-09-25 09:25:48 on dist-test-slave-4wm5"
server_key: "5f0edcc633be3fa7ade0e4a78b0b3f25"
server_key_iv: "c80022d9ffd64fd1c10ff6707a741c36"
server_key_version: "encryptionkey@0"
I20260925 09:25:48.931330 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal/instance:
uuid: "f1f75db3b3e343dd83aea3bbc9fabf42"
format_stamp: "Formatted at 2026-09-25 09:25:48 on dist-test-slave-4wm5"
server_key: "5f0edcc633be3fa7ade0e4a78b0b3f25"
server_key_iv: "c80022d9ffd64fd1c10ff6707a741c36"
server_key_version: "encryptionkey@0"
I20260925 09:25:48.937275 25976 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 09:25:48.941192 25991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:48.943235 25976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 09:25:48.943477 25976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal
uuid: "f1f75db3b3e343dd83aea3bbc9fabf42"
format_stamp: "Formatted at 2026-09-25 09:25:48 on dist-test-slave-4wm5"
server_key: "5f0edcc633be3fa7ade0e4a78b0b3f25"
server_key_iv: "c80022d9ffd64fd1c10ff6707a741c36"
server_key_version: "encryptionkey@0"
I20260925 09:25:48.943749 25976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:48.969544 25976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:48.975546 25976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:48.975891 25976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:48.997335 25976 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.60:43947
I20260925 09:25:48.997352 26042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.60:43947 every 8 connection(s)
I20260925 09:25:48.998881 25976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
I20260925 09:25:49.002741 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 25976
I20260925 09:25:49.002988 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790327973894689-3400-0/minicluster-data/master-2/wal/instance
I20260925 09:25:49.003368 26043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:49.003602 3400 external_mini_cluster.cc:1428] Setting key 7524f6ec1994158d87cace8da121150f
I20260925 09:25:49.014595 26043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:49.017872 25894 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 4
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:25:49.035697 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.005917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.3.82.60:43947)
0925 09:25:49.006969 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.006986 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.007034 (+ 48us) server_negotiation.cc:407] Connection header received
0925 09:25:49.009141 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.009177 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.009705 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.010299 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.012374 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.013788 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.015898 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.016758 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.016931 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.024422 (+ 7491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.024440 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.024477 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.024580 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.029509 (+ 4929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.030544 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.030554 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.030564 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.030814 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.031335 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.031342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.031347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.032014 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.032360 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.033022 (+ 662us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.033911 (+ 889us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":317,"server-negotiator.queue_time_us":796,"thread_start_us":476,"threads_started":1}
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:49.045059 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.018582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54540)
0925 09:25:49.019041 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:49.019068 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:49.019224 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:49.027485 (+ 8261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:49.027492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:49.027948 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:49.028120 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.028130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.030066 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.030073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.031446 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.031464 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.032532 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.032541 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.032762 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.033527 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:49.033548 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:49.036662 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:49.042045 (+ 5383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.042051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.042060 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.042724 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.043153 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.043159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.043163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.043313 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.043922 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:49.043930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:49.044100 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 09:25:49.044310 (+ 210us) client_negotiation.cc:240] Negotiation successful
0925 09:25:49.044643 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289}
I20260925 09:25:49.045115 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.019612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54540 (local address 127.3.82.60:43947)
0925 09:25:49.026916 (+ 7304us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.026924 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.026956 (+ 32us) server_negotiation.cc:407] Connection header received
0925 09:25:49.027031 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.027037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.027150 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.027294 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.028350 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.029908 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.031604 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.032311 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.032480 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.036903 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.036909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.036914 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.036952 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.041848 (+ 4896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.042870 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.042876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.042880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.043017 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.043431 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.043439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.043442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.043747 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.043914 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.044416 (+ 502us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.044650 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7108,"thread_start_us":6301,"threads_started":1}
I20260925 09:25:49.048312 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.018145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.3.82.62:43143)
0925 09:25:49.026581 (+ 8436us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.026588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.028720 (+ 2132us) server_negotiation.cc:407] Connection header received
0925 09:25:49.028988 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.028996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.029093 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.029217 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.031161 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.032243 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.035023 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.035510 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.035672 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.040588 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.040597 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.040605 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.040667 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.044479 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.045514 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.045519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.045523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.045670 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.046215 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.046223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.046229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.046654 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.046822 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.047506 (+ 684us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.047785 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8259,"thread_start_us":8110,"threads_started":1}
I20260925 09:25:49.048377 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.018135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:43143 (local address 127.0.0.1:37722)
0925 09:25:49.028443 (+ 10308us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:49.028490 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:49.028834 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:49.029269 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:49.029279 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:49.030309 (+ 1030us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:49.030776 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.030789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.032513 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.032518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.034822 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.034838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.035742 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.035749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.035960 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.036719 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:49.036768 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:49.040311 (+ 3543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:49.044667 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.044687 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.044696 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.045345 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.045878 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.045884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.045887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.046088 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.046881 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:49.046903 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:49.047117 (+ 214us) client_negotiation.cc:769] Sending connection context
0925 09:25:49.047340 (+ 223us) client_negotiation.cc:240] Negotiation successful
0925 09:25:49.047921 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9246,"mutex_wait_us":54,"thread_start_us":8380,"threads_started":1}
I20260925 09:25:49.050232 26043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:49.053303 25894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: Bootstrap starting.
I20260925 09:25:49.057588 25894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: Neither blocks nor log segments found. Creating new log.
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:49.059315 25894 log.cc:824] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:49.059883 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.039671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37734 (local address 127.3.82.62:43143)
0925 09:25:49.040266 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.040271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.040294 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:25:49.040345 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.040349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.040435 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.040550 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.042312 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.043855 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.045918 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.046519 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.046663 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.052019 (+ 5356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.052028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.052033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.052077 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.055999 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.056992 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.056998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.057001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.057190 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.057742 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.057749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.057752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.058232 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.058527 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.058936 (+ 409us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.059217 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":291,"threads_started":1}
I20260925 09:25:49.063458 25894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: No bootstrap required, opened a new log
I20260925 09:25:49.068377 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.051064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:42037 (local address 127.0.0.1:58294)
0925 09:25:49.051263 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:49.051283 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:49.051389 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:49.051858 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:49.051866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:49.052426 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:49.052631 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.052644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.054044 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.054050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.055535 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.055553 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.056542 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.056550 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.056790 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.057450 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:49.057470 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:49.060219 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:49.064692 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.064698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.064701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.065536 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.066088 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.066096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.066101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.066320 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.067251 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:49.067256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:49.067463 (+ 207us) client_negotiation.cc:769] Sending connection context
0925 09:25:49.067652 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 09:25:49.068006 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 09:25:49.068500 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.051294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.3.82.61:42037)
0925 09:25:49.051489 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.051497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.051518 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:25:49.051588 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.051594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.051686 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.051801 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.052821 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.053874 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.055717 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.056345 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.056497 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.060494 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.060500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.060503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.060542 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.064493 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.065685 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.065693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.065696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.065923 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.066465 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.066470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.066474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.066850 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.067153 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.067794 (+ 641us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.068067 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:25:49.070147 26043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:49.074419 25894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.074812 25894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068106a6633c4553954b4c813e88384e, State: Initialized, Role: FOLLOWER
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:49.075409 25894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.076511 26053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.077044 26053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.078017 25894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:25:49.086220 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.062211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.3.82.61:42037)
0925 09:25:49.064824 (+ 2613us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.064830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.064852 (+ 22us) server_negotiation.cc:407] Connection header received
0925 09:25:49.064919 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.064924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.064999 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.065112 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.067232 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.068555 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.070509 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.071196 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.071968 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.077045 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.077053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.077058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.077114 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.081358 (+ 4244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.082590 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.082596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.082602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.082790 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.083327 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.083330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.083332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.083780 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.085292 (+ 1512us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.085435 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.085698 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2476,"thread_start_us":369,"threads_started":1}
I20260925 09:25:49.088912 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.071947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54546)
0925 09:25:49.072206 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:49.072225 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:49.072347 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:49.072987 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:49.072992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:49.073502 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:49.073701 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.073713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.075193 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.075201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.076402 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.076422 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.077602 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.077610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.077919 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.078530 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:49.078553 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:49.081563 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:49.085967 (+ 4404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.085974 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.085980 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.086526 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.086963 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.086968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.086971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.087116 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.087842 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:49.087847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:49.088033 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 09:25:49.088180 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 09:25:49.088517 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260925 09:25:49.089926 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.072447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54546 (local address 127.3.82.60:43947)
0925 09:25:49.072627 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.072632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.072652 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:25:49.072703 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.072708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.072807 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.072943 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.073924 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.075055 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.076580 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.077184 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.077326 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.081924 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.081937 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.081946 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.082005 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.085768 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.086681 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.086688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.086693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.086834 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.087249 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.087253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.087256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.087712 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.089166 (+ 1454us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.089298 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.089539 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:25:49.093685 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:49.096066 26043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: Bootstrap starting.
I20260925 09:25:49.098124 25869 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "068106a6633c4553954b4c813e88384e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37734
I20260925 09:25:49.098650 25869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 09:25:49.098929 25869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:49.100780 26043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: Neither blocks nor log segments found. Creating new log.
W20260925 09:25:49.102137 25848 tablet.cc:2369] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:25:49.102330 26043 log.cc:824] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:49.088835 25827 master_runner.cc:186] Error getting master registration for 127.3.82.61:42037: OK, instance_id { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" instance_seqno: 1790328348437045 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:25:49.102631 25827 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:25:49.104245 25869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
W20260925 09:25:49.104511 26064 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:25:49.104614 26064 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:25:49.105496 26043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: No bootstrap required, opened a new log
I20260925 09:25:49.106017 25869 leader_election.cc:288] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [CANDIDATE]: Term 1 election: Requested vote from peers 4d916084094e4e22b5fc4e47cad671a0 (127.3.82.61:42037), f1f75db3b3e343dd83aea3bbc9fabf42 (127.3.82.60:43947)
I20260925 09:25:49.120836 26043 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.121526 26043 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1f75db3b3e343dd83aea3bbc9fabf42, State: Initialized, Role: FOLLOWER
I20260925 09:25:49.122215 26043 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.123909 26071 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.124965 26071 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.125437 26043 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:49.125496 26018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "068106a6633c4553954b4c813e88384e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42"
I20260925 09:25:49.126084 26018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:49.130069 26018 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068106a6633c4553954b4c813e88384e in term 1.
I20260925 09:25:49.133533 25843 leader_election.cc:302] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068106a6633c4553954b4c813e88384e, f1f75db3b3e343dd83aea3bbc9fabf42; no voters:
I20260925 09:25:49.134434 26053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:49.162556 26053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [term 1 LEADER]: Becoming Leader. State: Replica: 068106a6633c4553954b4c813e88384e, State: Running, Role: LEADER
I20260925 09:25:49.164259 26053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.169728 25968 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } attempt: 5
I20260925 09:25:49.174901 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.131058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.3.82.61:42037)
0925 09:25:49.131431 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.131441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.150001 (+ 18560us) server_negotiation.cc:407] Connection header received
0925 09:25:49.155870 (+ 5869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.155877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.156019 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.156109 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.158539 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.159614 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.161603 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.162358 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.162472 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.166665 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.166672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.166677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.166731 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.170571 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.171599 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.171607 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.171612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.172056 (+ 444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.172573 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.172578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.172584 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.173015 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.173252 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.174205 (+ 953us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.174457 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
Sep 25 09:25:49 dist-test-slave-4wm5 krb5kdc[25814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328348, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:25:49.190464 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.131442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37744 (local address 127.3.82.62:43143)
0925 09:25:49.131692 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.131701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.140899 (+ 9198us) server_negotiation.cc:407] Connection header received
0925 09:25:49.150670 (+ 9771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.150678 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.150786 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.150878 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.162781 (+ 11903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.163970 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.169936 (+ 5966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.170706 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.170811 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.182920 (+ 12109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.182928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.182932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.182984 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.186761 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.187753 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.187759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.187763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.187935 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.188427 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.188432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.188435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.188852 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.189216 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.189808 (+ 592us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.190107 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 09:25:49.199718 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.149236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54556 (local address 127.3.82.60:43947)
0925 09:25:49.149490 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.149499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.153740 (+ 4241us) server_negotiation.cc:407] Connection header received
0925 09:25:49.154312 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.154318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.154456 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.154536 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.166555 (+ 12019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.167702 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.171825 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.172633 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.172735 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.182951 (+ 10216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.182958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.182961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.183005 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.193984 (+ 10979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.195549 (+ 1565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.195558 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.195564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.195737 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.196545 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.196550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.196582 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.197081 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.197384 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.198551 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.199004 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"spinlock_wait_cycles":30080}
I20260925 09:25:49.201063 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.170433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:43947 (local address 127.0.0.1:54560)
0925 09:25:49.170899 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:49.170924 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:49.171062 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:49.171819 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:49.171824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:49.172467 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:49.172770 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.172786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.174795 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.174805 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.176869 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:49.176890 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.178636 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.178646 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:49.179104 (+ 458us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.180165 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:49.180221 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:49.185736 (+ 5515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:49.197042 (+ 11306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.197052 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.197059 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.197817 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.198392 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:49.198397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:49.198404 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:49.198647 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:49.199618 (+ 971us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:49.199627 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:49.199867 (+ 240us) client_negotiation.cc:769] Sending connection context
0925 09:25:49.200088 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 09:25:49.200476 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301}
I20260925 09:25:49.202601 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:49.171318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.3.82.60:43947)
0925 09:25:49.171479 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.171486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.171507 (+ 21us) server_negotiation.cc:407] Connection header received
0925 09:25:49.171574 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.171581 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.171670 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.171787 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.172991 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.174544 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.177154 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:49.178116 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:49.178275 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:49.186130 (+ 7855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:49.186140 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:49.186148 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:49.186203 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:49.196801 (+ 10598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.197991 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.198000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.198007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.198210 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:49.198808 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:49.198816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:49.198822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:49.199306 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:49.199509 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:49.201573 (+ 2064us) server_negotiation.cc:299] Negotiation successful
0925 09:25:49.202010 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 09:25:49.209015 26075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 068106a6633c4553954b4c813e88384e. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.209551 26075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:49.209908 25968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: Bootstrap starting.
I20260925 09:25:49.216871 25968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:49.218636 25968 log.cc:824] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:49.220141 26080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:49.227434 25968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: No bootstrap required, opened a new log
W20260925 09:25:49.228729 25997 tablet.cc:2369] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:25:49.234272 26080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:49.253037 25968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.253636 25968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d916084094e4e22b5fc4e47cad671a0, State: Initialized, Role: FOLLOWER
I20260925 09:25:49.254336 25968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } }
I20260925 09:25:49.255864 26018 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [term 1 FOLLOWER]: Refusing update from remote peer 068106a6633c4553954b4c813e88384e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:49.257015 26075 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:49.273561 25942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:49.274467 25968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:49.273245 25943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "068106a6633c4553954b4c813e88384e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4d916084094e4e22b5fc4e47cad671a0"
I20260925 09:25:49.276566 26071 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 068106a6633c4553954b4c813e88384e. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.276932 26071 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.278738 25942 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [term 1 FOLLOWER]: Refusing update from remote peer 068106a6633c4553954b4c813e88384e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:49.280076 26053 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:49.265046 26090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.287230 26090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.287477 25943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068106a6633c4553954b4c813e88384e in term 1.
I20260925 09:25:49.299731 26087 mvcc.cc:205] Tried to move back new op lower bound from 7333184918537818112 to 7333184918191607808. Current Snapshot: MvccSnapshot[applied={T|T < 7333184918537818112}]
I20260925 09:25:49.322407 26080 catalog_manager.cc:1347] Generated new cluster ID: 00a9975f0c78450a9e6739474cc12312
I20260925 09:25:49.322546 26080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:49.342741 26075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.343292 26075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:49.345271 26071 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.345574 26071 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.346551 26091 mvcc.cc:205] Tried to move back new op lower bound from 7333184918537818112 to 7333184918191607808. Current Snapshot: MvccSnapshot[applied={T|T < 7333184918537818112 or (T in {7333184918537818112})}]
I20260925 09:25:49.374188 26080 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:49.376937 26080 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:49.382974 26090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 068106a6633c4553954b4c813e88384e. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.383311 26090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.385669 26053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.386034 26053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:49.391978 26100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "068106a6633c4553954b4c813e88384e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068106a6633c4553954b4c813e88384e" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 43143 } } peers { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 42037 } } peers { permanent_uuid: "f1f75db3b3e343dd83aea3bbc9fabf42" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 43947 } } }
I20260925 09:25:49.392616 26100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:49.382113 25976 master_runner.cc:186] Error getting master registration for 127.3.82.61:42037: OK, instance_id { permanent_uuid: "4d916084094e4e22b5fc4e47cad671a0" instance_seqno: 1790328348437045 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:25:49.397930 25976 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:25:49.417533 26093 catalog_manager.cc:1259] Loaded cluster ID: 00a9975f0c78450a9e6739474cc12312
I20260925 09:25:49.417650 26093 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: loading cluster ID for follower catalog manager: success
I20260925 09:25:49.421500 26080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 068106a6633c4553954b4c813e88384e: Generated new TSK 0
I20260925 09:25:49.422730 26080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:25:49.426819 26093 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: acquiring CA information for follower catalog manager: success
I20260925 09:25:49.432260 26093 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f1f75db3b3e343dd83aea3bbc9fabf42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:25:49.438119 26113 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260925 09:25:49.528805 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.519327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37750 (local address 127.3.82.62:43143)
0925 09:25:49.519742 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.519748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.526598 (+ 6850us) server_negotiation.cc:407] Connection header received
0925 09:25:49.526957 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.526965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.527085 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.527174 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.528491 (+ 1317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37750: BlockingRecv error: recv got EOF from 127.0.0.1:37750 (error 108)
Metrics: {"server-negotiator.queue_time_us":282}
W20260925 09:25:49.531634 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.519538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.3.82.61:42037)
0925 09:25:49.519716 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.519722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.529738 (+ 10016us) server_negotiation.cc:407] Connection header received
0925 09:25:49.530047 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.530054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.530154 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.530231 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.531342 (+ 1111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58328: BlockingRecv error: recv got EOF from 127.0.0.1:58328 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 09:25:49.534539 26046 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.532186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.3.82.60:43947)
0925 09:25:49.532384 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.532389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.532706 (+ 317us) server_negotiation.cc:407] Connection header received
0925 09:25:49.533074 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.533081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.533177 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.533359 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.534252 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54576: BlockingRecv error: recv got EOF from 127.0.0.1:54576 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 09:25:49.542260 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.539240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.3.82.61:42037)
0925 09:25:49.539437 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.539444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.540428 (+ 984us) server_negotiation.cc:407] Connection header received
0925 09:25:49.540677 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.540683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.540832 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.540986 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.541990 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58344: BlockingRecv error: recv got EOF from 127.0.0.1:58344 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 09:25:49.549788 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.547325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58360 (local address 127.3.82.61:42037)
0925 09:25:49.547509 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.547516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.548142 (+ 626us) server_negotiation.cc:407] Connection header received
0925 09:25:49.548367 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.548374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.548473 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.548594 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.549517 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58360: BlockingRecv error: recv got EOF from 127.0.0.1:58360 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260925 09:25:49.558116 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.555857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.3.82.61:42037)
0925 09:25:49.556082 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.556092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.556603 (+ 511us) server_negotiation.cc:407] Connection header received
0925 09:25:49.556802 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.556808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.556913 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.557023 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.557875 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58374: BlockingRecv error: recv got EOF from 127.0.0.1:58374 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260925 09:25:49.574332 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.571828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.3.82.61:42037)
0925 09:25:49.572083 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.572091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.572634 (+ 543us) server_negotiation.cc:407] Connection header received
0925 09:25:49.572868 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.572875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.572973 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.573089 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.574032 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58380: BlockingRecv error: recv got EOF from 127.0.0.1:58380 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
W20260925 09:25:49.588936 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.586491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.3.82.61:42037)
0925 09:25:49.586705 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.586711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.587243 (+ 532us) server_negotiation.cc:407] Connection header received
0925 09:25:49.587483 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.587489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.587622 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.587737 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.588698 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58390: BlockingRecv error: recv got EOF from 127.0.0.1:58390 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260925 09:25:49.611481 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.609068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.3.82.61:42037)
0925 09:25:49.609285 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.609292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.609878 (+ 586us) server_negotiation.cc:407] Connection header received
0925 09:25:49.610115 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.610121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.610217 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.610331 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.611224 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58396: BlockingRecv error: recv got EOF from 127.0.0.1:58396 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260925 09:25:49.651396 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.648872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.3.82.61:42037)
0925 09:25:49.649103 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.649109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.649686 (+ 577us) server_negotiation.cc:407] Connection header received
0925 09:25:49.649920 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.649926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.650025 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.650145 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.651116 (+ 971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58410: BlockingRecv error: recv got EOF from 127.0.0.1:58410 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260925 09:25:49.721136 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.719196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.3.82.61:42037)
0925 09:25:49.719425 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.719432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.719742 (+ 310us) server_negotiation.cc:407] Connection header received
0925 09:25:49.719901 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.719908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.720016 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.720156 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.720899 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58426: BlockingRecv error: recv got EOF from 127.0.0.1:58426 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260925 09:25:49.855132 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:49.852952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.3.82.61:42037)
0925 09:25:49.853165 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:49.853169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:49.853573 (+ 404us) server_negotiation.cc:407] Connection header received
0925 09:25:49.853775 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:49.853779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:49.853942 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:49.854046 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:49.854956 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58428: BlockingRecv error: recv got EOF from 127.0.0.1:58428 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260925 09:25:50.117980 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:50.115954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.3.82.61:42037)
0925 09:25:50.116249 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:50.116256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:50.116426 (+ 170us) server_negotiation.cc:407] Connection header received
0925 09:25:50.116662 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:50.116667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:50.116751 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:50.116864 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:50.117674 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58436: BlockingRecv error: recv got EOF from 127.0.0.1:58436 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
I20260925 09:25:50.442837 26113 catalog_manager.cc:1259] Loaded cluster ID: 00a9975f0c78450a9e6739474cc12312
I20260925 09:25:50.443012 26113 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: loading cluster ID for follower catalog manager: success
I20260925 09:25:50.446022 26113 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: acquiring CA information for follower catalog manager: success
I20260925 09:25:50.448334 26113 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d916084094e4e22b5fc4e47cad671a0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:25:50.643416 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:50.635292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.3.82.61:42037)
0925 09:25:50.636207 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:50.636213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:50.636238 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:25:50.636321 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:50.636326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:50.636536 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:50.636697 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:50.637375 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.638725 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.640212 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.640990 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.641140 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:50.641766 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:50.642609 (+ 843us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:50.642780 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:50.642918 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 09:25:50.643034 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":566,"threads_started":1}
I20260925 09:25:50.659149 26184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:50.650346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.3.82.60:43947)
0925 09:25:50.651095 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:50.651103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:50.651130 (+ 27us) server_negotiation.cc:407] Connection header received
0925 09:25:50.651402 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:50.651418 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:50.651721 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:50.651949 (+ 228us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:50.652503 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.653633 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.655293 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.655966 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.656116 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:50.656822 (+ 706us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:50.657547 (+ 725us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:50.657726 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:50.658001 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 09:25:50.658123 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":500,"threads_started":1}
I20260925 09:25:50.681128 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:50.668075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37752 (local address 127.3.82.62:43143)
0925 09:25:50.668603 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:50.668613 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:50.670484 (+ 1871us) server_negotiation.cc:407] Connection header received
0925 09:25:50.670864 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:50.670871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:50.671140 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:50.671347 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:50.672599 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.674186 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.677886 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.678580 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.678702 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:50.679645 (+ 943us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:50.680195 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:50.680328 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:50.680695 (+ 367us) server_negotiation.cc:299] Negotiation successful
0925 09:25:50.680803 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":254,"threads_started":1}
I20260925 09:25:50.686746 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:50.667878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.3.82.61:42037)
0925 09:25:50.668089 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:50.668095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:50.678558 (+ 10463us) server_negotiation.cc:407] Connection header received
0925 09:25:50.678935 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:50.678944 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:50.679080 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:50.679156 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:50.680256 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.681394 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.684212 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.684936 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.685062 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:50.685799 (+ 737us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:50.685961 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:50.686037 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:50.686286 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 09:25:50.686376 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 09:25:50.686976 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25827
I20260925 09:25:50.688747 26184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:50.671024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.3.82.60:43947)
0925 09:25:50.671709 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:50.671718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:50.681527 (+ 9809us) server_negotiation.cc:407] Connection header received
0925 09:25:50.681601 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:50.681605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:50.681675 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:50.681800 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:50.682940 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.684045 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.686263 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:50.686923 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:50.687033 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:50.687824 (+ 791us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:50.687983 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:50.688067 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:50.688311 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 09:25:50.688415 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513}
I20260925 09:25:50.690927 25889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:50.807898 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25827
I20260925 09:25:50.830268 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25901
I20260925 09:25:50.831523 25963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:50.944518 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25901
I20260925 09:25:50.965689 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 25976
I20260925 09:25:50.966895 26038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:51.077922 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 25976
2026-09-25T09:25:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5133 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 09:25:51 dist-test-slave-4wm5 krb5kdc[26198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 09:25:51 dist-test-slave-4wm5 krb5kdc[26198](info): set up 2 sockets
Sep 25 09:25:51 dist-test-slave-4wm5 krb5kdc[26198](info): commencing operation
krb5kdc: starting...
W20260925 09:25:52.230556 3400 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/test-user.keytab.
Sep 25 09:25:52 dist-test-slave-4wm5 krb5kdc[26198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T09:25:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:25:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.82.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
Entry for principal kudu/127.3.82.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab.
I20260925 09:25:52.527253 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:35687
--webserver_interface=127.3.82.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:35687,127.3.82.61:39857,127.3.82.60:44871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 09:25:52.884336 26211 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:25:52.884748 26211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:52.884873 26211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:52.885002 26211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:52.892765 26211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:52.892880 26211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:52.892979 26211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:52.893057 26211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:52.893167 26211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:52.904527 26211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:35687,127.3.82.61:39857,127.3.82.60:44871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.62.keytab
--principal=kudu/127.3.82.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.62:35687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.82.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:52.906193 26211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:52.907806 26211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:52.921314 26219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:52.921438 26217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:52.922034 26216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:52.922477 26211 server_base.cc:1034] running on GCE node
I20260925 09:25:52.923379 26211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:52.924953 26211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:52.926198 26211 hybrid_clock.cc:648] HybridClock initialized: now 1790328352926145 us; error 40 us; skew 500 ppm
I20260925 09:25:52.926553 26211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:52 dist-test-slave-4wm5 krb5kdc[26198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:52.935631 26211 init.cc:377] Logged in from keytab as kudu/127.3.82.62@KRBTEST.COM (short username kudu)
I20260925 09:25:52.937601 26211 webserver.cc:460] Webserver started at http://127.3.82.62:36299/ using document root <none> and password file <none>
I20260925 09:25:52.938452 26211 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:52.938593 26211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:52.938949 26211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:52.942273 26211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data/instance:
uuid: "9dfcb774ce98414696fcd1dcea831114"
format_stamp: "Formatted at 2026-09-25 09:25:52 on dist-test-slave-4wm5"
server_key: "96c6d728bebb355fa3e8cf452f75b90c"
server_key_iv: "261216ece6c09501bc0bbf6d73ca8167"
server_key_version: "encryptionkey@0"
I20260925 09:25:52.943164 26211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal/instance:
uuid: "9dfcb774ce98414696fcd1dcea831114"
format_stamp: "Formatted at 2026-09-25 09:25:52 on dist-test-slave-4wm5"
server_key: "96c6d728bebb355fa3e8cf452f75b90c"
server_key_iv: "261216ece6c09501bc0bbf6d73ca8167"
server_key_version: "encryptionkey@0"
I20260925 09:25:52.949095 26211 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260925 09:25:52.953176 26226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:52.954774 26211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:25:52.955102 26211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal
uuid: "9dfcb774ce98414696fcd1dcea831114"
format_stamp: "Formatted at 2026-09-25 09:25:52 on dist-test-slave-4wm5"
server_key: "96c6d728bebb355fa3e8cf452f75b90c"
server_key_iv: "261216ece6c09501bc0bbf6d73ca8167"
server_key_version: "encryptionkey@0"
I20260925 09:25:52.955365 26211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:52.970314 26211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:52.975658 26211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:52.976033 26211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:52.996716 26211 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.62:35687
I20260925 09:25:52.996737 26277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.62:35687 every 8 connection(s)
I20260925 09:25:52.998410 26211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/data/info.pb
I20260925 09:25:53.002872 26278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:53.006495 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 26211
I20260925 09:25:53.006705 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-0/wal/instance
I20260925 09:25:53.007293 3400 external_mini_cluster.cc:1428] Setting key bcecfd0294911f7589c2e56f055f9326
I20260925 09:25:53.012822 26278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:53 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.61@KRBTEST.COM" created.
I20260925 09:25:53.038906 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.017742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.3.82.62:35687)
0925 09:25:53.018889 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:53.018905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:53.019085 (+ 180us) server_negotiation.cc:407] Connection header received
0925 09:25:53.021330 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:53.021337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:53.021619 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:53.021891 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:53.023353 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.024676 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.026221 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.026791 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.027001 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.030556 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:53.030564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:53.030578 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:53.030607 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:53.033815 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.034680 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.034693 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.034700 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.034920 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.035423 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.035427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.035429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.036227 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:53.036439 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:53.037169 (+ 730us) server_negotiation.cc:299] Negotiation successful
0925 09:25:53.037765 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":226,"server-negotiator.queue_time_us":936,"spinlock_wait_cycles":27392,"thread_start_us":672,"threads_started":1}
I20260925 09:25:53.038931 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.017742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35687 (local address 127.0.0.1:48362)
0925 09:25:53.018871 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.018920 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:53.020886 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:53.021898 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:53.021906 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:53.022706 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:53.023078 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.023098 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.024851 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.024858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.026074 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.026088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.027042 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.027049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.027211 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.027968 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:53.027996 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:53.030297 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:53.034037 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.034050 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.034064 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.034471 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.035062 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.035067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.035070 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.035252 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.036398 (+ 1146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:53.036405 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:53.036651 (+ 246us) client_negotiation.cc:769] Sending connection context
0925 09:25:53.036953 (+ 302us) client_negotiation.cc:240] Negotiation successful
0925 09:25:53.037499 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":417,"threads_started":1}
I20260925 09:25:53.042573 26278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:53.043972 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.043382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35338)
0925 09:25:53.043637 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.043782 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260925 09:25:53.044314 26227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
I20260925 09:25:53.046350 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.045788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35344)
0925 09:25:53.046041 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.046147 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260925 09:25:53.046675 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:39857: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab.
Entry for principal kudu/127.3.82.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab.
I20260925 09:25:53.073992 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:39857
--webserver_interface=127.3.82.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab
--principal=kudu/127.3.82.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:35687,127.3.82.61:39857,127.3.82.60:44871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:25:53.096174 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } attempt: 1
I20260925 09:25:53.097699 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.097147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35352)
0925 09:25:53.097391 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.097488 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260925 09:25:53.098893 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.098479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35358)
0925 09:25:53.098660 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.098741 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 09:25:53.099100 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:39857: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
I20260925 09:25:53.167373 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } attempt: 2
I20260925 09:25:53.168716 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.168114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35370)
0925 09:25:53.168399 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.168501 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260925 09:25:53.170302 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.169774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35378)
0925 09:25:53.170021 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.170122 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260925 09:25:53.170524 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:39857: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
I20260925 09:25:53.261798 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } attempt: 3
I20260925 09:25:53.263124 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.262619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35390)
0925 09:25:53.262829 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.262932 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 09:25:53.264286 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.263893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35392)
0925 09:25:53.264067 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.264139 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:25:53.264472 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:39857: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
I20260925 09:25:53.407748 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } attempt: 4
I20260925 09:25:53.409495 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.408669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35398)
0925 09:25:53.409072 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.409238 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260925 09:25:53.410765 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.410372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35412)
0925 09:25:53.410541 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.410627 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260925 09:25:53.411010 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.61:39857: Network error: Client connection negotiation failed: client connection to 127.3.82.61:39857: connect: Connection refused (error 111)
W20260925 09:25:53.446864 26285 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:25:53.447218 26285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:53.447286 26285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:53.447362 26285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:53.455273 26285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:53.455392 26285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:53.455449 26285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:53.455494 26285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:53.455557 26285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:53.466435 26285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:35687,127.3.82.61:39857,127.3.82.60:44871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.61.keytab
--principal=kudu/127.3.82.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.61:39857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.82.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:53.467831 26285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:53.469314 26285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:53.482132 26290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:53.482164 26293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:53.482357 26291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:53.487843 26285 server_base.cc:1034] running on GCE node
I20260925 09:25:53.488862 26285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:53.490489 26285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:53.491765 26285 hybrid_clock.cc:648] HybridClock initialized: now 1790328353491676 us; error 61 us; skew 500 ppm
I20260925 09:25:53.492152 26285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:53 dist-test-slave-4wm5 krb5kdc[26198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328353, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:53.495551 26285 init.cc:377] Logged in from keytab as kudu/127.3.82.61@KRBTEST.COM (short username kudu)
I20260925 09:25:53.497169 26285 webserver.cc:460] Webserver started at http://127.3.82.61:40537/ using document root <none> and password file <none>
I20260925 09:25:53.497959 26285 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:53.498056 26285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:53.498318 26285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:53.501430 26285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data/instance:
uuid: "4dabf75a321b4816b44958a67a8feeac"
format_stamp: "Formatted at 2026-09-25 09:25:53 on dist-test-slave-4wm5"
server_key: "af94ba92e929e7ab409afb596277b738"
server_key_iv: "ca038dc70547810755d7371a74cafb88"
server_key_version: "encryptionkey@0"
I20260925 09:25:53.502305 26285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal/instance:
uuid: "4dabf75a321b4816b44958a67a8feeac"
format_stamp: "Formatted at 2026-09-25 09:25:53 on dist-test-slave-4wm5"
server_key: "af94ba92e929e7ab409afb596277b738"
server_key_iv: "ca038dc70547810755d7371a74cafb88"
server_key_version: "encryptionkey@0"
I20260925 09:25:53.508200 26285 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 09:25:53.512502 26300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:53.514144 26285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:25:53.514366 26285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal
uuid: "4dabf75a321b4816b44958a67a8feeac"
format_stamp: "Formatted at 2026-09-25 09:25:53 on dist-test-slave-4wm5"
server_key: "af94ba92e929e7ab409afb596277b738"
server_key_iv: "ca038dc70547810755d7371a74cafb88"
server_key_version: "encryptionkey@0"
I20260925 09:25:53.514632 26285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:53.534381 26285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:53.539862 26285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:53.540233 26285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:53.563114 26285 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.61:39857
I20260925 09:25:53.563303 26351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.61:39857 every 8 connection(s)
I20260925 09:25:53.564669 26285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/data/info.pb
I20260925 09:25:53.568807 26352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:53.572929 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 26285
I20260925 09:25:53.573165 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-1/wal/instance
I20260925 09:25:53.573762 3400 external_mini_cluster.cc:1428] Setting key 85be90b8c303cd816ab0d173485d9d12
I20260925 09:25:53.577792 26352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:53 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328353, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.82.60@KRBTEST.COM; defaulting to no policy
I20260925 09:25:53.601707 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.581755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.3.82.62:35687)
0925 09:25:53.582403 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:53.582413 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:53.583633 (+ 1220us) server_negotiation.cc:407] Connection header received
0925 09:25:53.586204 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:53.586210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:53.586312 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:53.586446 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:53.588221 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.588993 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.590720 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.591156 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.591305 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.595293 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:53.595298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:53.595301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:53.595343 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:53.598344 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.599208 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.599213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.599215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.599339 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.599821 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.599825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.599827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.600123 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:53.600282 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:53.601057 (+ 775us) server_negotiation.cc:299] Negotiation successful
0925 09:25:53.601296 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":307,"threads_started":1}
I20260925 09:25:53.602789 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.582517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35687 (local address 127.0.0.1:48372)
0925 09:25:53.583395 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.583461 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:53.585807 (+ 2346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:53.586719 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:53.586729 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:53.587331 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:53.587964 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.587983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.589178 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.589182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.590554 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.590568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.591408 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.591415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.591674 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.592429 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:53.592454 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:53.594903 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:53.598527 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.598539 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.598562 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.599052 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.599464 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.599470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.599474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.599683 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.600276 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:53.600285 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:53.600672 (+ 387us) client_negotiation.cc:769] Sending connection context
0925 09:25:53.600956 (+ 284us) client_negotiation.cc:240] Negotiation successful
0925 09:25:53.601538 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":322,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.82.60@KRBTEST.COM" created.
I20260925 09:25:53.604750 26352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:53 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328353, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:53.621014 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.605486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35418)
0925 09:25:53.605702 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.605722 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:53.605816 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:53.607123 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:53.607127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:53.607622 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:53.607756 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.607765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.609301 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.609307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.610217 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.610229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.611086 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.611091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.611286 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.611978 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:53.611995 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:53.614182 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:53.618012 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.618018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.618023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.618469 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.619010 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.619014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.619017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.619145 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.619817 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:53.619822 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:53.620016 (+ 194us) client_negotiation.cc:769] Sending connection context
0925 09:25:53.620184 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 09:25:53.620534 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 09:25:53.621145 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.605898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.3.82.61:39857)
0925 09:25:53.606689 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:53.606696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:53.606722 (+ 26us) server_negotiation.cc:407] Connection header received
0925 09:25:53.606774 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:53.606780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:53.606968 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:53.607137 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:53.607941 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.609149 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.610356 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.610925 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.611070 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.614400 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:53.614414 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:53.614430 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:53.614459 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:53.617777 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.618645 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.618658 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.618665 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.618807 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.619273 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.619280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.619289 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.619676 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:53.619929 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:53.620359 (+ 430us) server_negotiation.cc:299] Negotiation successful
0925 09:25:53.620672 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":634,"thread_start_us":486,"threads_started":1}
I20260925 09:25:53.624367 26352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:53.626006 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.625427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:52974)
0925 09:25:53.625626 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.625764 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260925 09:25:53.626345 26302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.628098 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.627694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:52976)
0925 09:25:53.627888 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.627970 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:25:53.628363 26352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.82.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab.
Entry for principal kudu/127.3.82.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab.
I20260925 09:25:53.642652 3400 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0mddRs/build/asan/bin/kudu
/tmp/dist-test-task0mddRs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:44871
--webserver_interface=127.3.82.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab
--principal=kudu/127.3.82.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.82.20:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.82.62:35687,127.3.82.61:39857,127.3.82.60:44871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 09:25:53.677706 26352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 1
I20260925 09:25:53.679050 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.678509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:52982)
0925 09:25:53.678741 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.678862 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 09:25:53.680176 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.679777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:52996)
0925 09:25:53.679941 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.680047 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 09:25:53.680445 26352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.710301 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } attempt: 5
Sep 25 09:25:53 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:53.727793 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.711146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35424)
0925 09:25:53.711355 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.711378 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:53.711505 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:53.712056 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:53.712061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:53.712458 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:53.712633 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.712642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.714430 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.714437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.715413 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:53.715429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.716573 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.716579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:53.716749 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.717620 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:53.717640 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:53.720779 (+ 3139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:53.725098 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.725102 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.725110 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.725545 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.726082 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:53.726085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:53.726088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:53.726229 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:53.726904 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:53.726907 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:53.727124 (+ 217us) client_negotiation.cc:769] Sending connection context
0925 09:25:53.727255 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 09:25:53.727529 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 09:25:53.727959 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.711261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.3.82.61:39857)
0925 09:25:53.711549 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:53.711556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:53.711581 (+ 25us) server_negotiation.cc:407] Connection header received
0925 09:25:53.711653 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:53.711659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:53.711764 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:53.711908 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:53.712830 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.714224 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.715606 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:53.716372 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:53.716532 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:53.721058 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:53.721070 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:53.721077 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:53.721143 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:53.724890 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.725703 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.725712 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.725717 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.725945 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:53.726346 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:53.726352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:53.726364 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:53.726770 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:53.726913 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:53.727388 (+ 475us) server_negotiation.cc:299] Negotiation successful
0925 09:25:53.727657 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:25:53.729105 26278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:53.730297 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.729817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53006)
0925 09:25:53.730053 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.730147 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260925 09:25:53.731283 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.730931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53012)
0925 09:25:53.731107 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.731173 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260925 09:25:53.731549 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.748718 26352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 2
I20260925 09:25:53.750038 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.749492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53028)
0925 09:25:53.749700 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.749804 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260925 09:25:53.751178 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.750799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53038)
0925 09:25:53.750971 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.751055 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260925 09:25:53.751461 26352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.782819 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 1
I20260925 09:25:53.784282 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.783672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53052)
0925 09:25:53.783981 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.784092 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260925 09:25:53.785378 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.785043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53064)
0925 09:25:53.785192 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.785271 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 09:25:53.785598 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.842746 26352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 3
I20260925 09:25:53.844482 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.843753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53070)
0925 09:25:53.844156 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.844292 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260925 09:25:53.846122 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.845555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53076)
0925 09:25:53.845808 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.845986 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260925 09:25:53.846342 26352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.853902 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 2
I20260925 09:25:53.855187 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.854666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53080)
0925 09:25:53.854899 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.855011 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:25:53.856289 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.855926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53092)
0925 09:25:53.856096 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.856173 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260925 09:25:53.856573 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.962816 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 3
I20260925 09:25:53.964222 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.963578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53096)
0925 09:25:53.963860 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.963994 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260925 09:25:53.965658 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.965131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53110)
0925 09:25:53.965380 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.965484 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260925 09:25:53.965965 26278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
I20260925 09:25:53.989714 26352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 4
I20260925 09:25:53.991142 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.990539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53122)
0925 09:25:53.990808 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.990900 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 09:25:53.992381 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:53.991935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53134)
0925 09:25:53.992189 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:53.992263 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260925 09:25:53.992801 26352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.82.60:44871: Network error: Client connection negotiation failed: client connection to 127.3.82.60:44871: connect: Connection refused (error 111)
W20260925 09:25:54.015233 26360 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 09:25:54.015594 26360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:25:54.015659 26360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:25:54.015746 26360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:25:54.023815 26360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:25:54.023931 26360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 09:25:54.023989 26360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:25:54.024032 26360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:25:54.024094 26360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:25:54.034598 26360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.82.20:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.82.62:35687,127.3.82.61:39857,127.3.82.60:44871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/krb5kdc/kudu_127.3.82.60.keytab
--principal=kudu/127.3.82.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.82.60:44871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.82.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 07:45:44 UTC on ccycloud.quasar-yomzrf.root.comops.site
build id 4671939
ASAN enabled
I20260925 09:25:54.035923 26360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:25:54.037259 26360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:25:54.051394 26368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:54.051646 26366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:25:54.051421 26365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:25:54.052554 26360 server_base.cc:1034] running on GCE node
I20260925 09:25:54.053519 26360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:25:54.055330 26360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:25:54.056641 26360 hybrid_clock.cc:648] HybridClock initialized: now 1790328354056589 us; error 51 us; skew 500 ppm
I20260925 09:25:54.057071 26360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328354, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 09:25:54.061277 26360 init.cc:377] Logged in from keytab as kudu/127.3.82.60@KRBTEST.COM (short username kudu)
I20260925 09:25:54.063308 26360 webserver.cc:460] Webserver started at http://127.3.82.60:40595/ using document root <none> and password file <none>
I20260925 09:25:54.064110 26360 fs_manager.cc:359] Metadata directory not provided
I20260925 09:25:54.064216 26360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:25:54.064540 26360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:25:54.068143 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data/instance:
uuid: "d4c204c1ced94ce5b99a2ebafd4f7470"
format_stamp: "Formatted at 2026-09-25 09:25:54 on dist-test-slave-4wm5"
server_key: "70d24232f68f14e2822076576fb7baa9"
server_key_iv: "5461b55a52a3676d9ee57cb007ba8420"
server_key_version: "encryptionkey@0"
I20260925 09:25:54.069272 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal/instance:
uuid: "d4c204c1ced94ce5b99a2ebafd4f7470"
format_stamp: "Formatted at 2026-09-25 09:25:54 on dist-test-slave-4wm5"
server_key: "70d24232f68f14e2822076576fb7baa9"
server_key_iv: "5461b55a52a3676d9ee57cb007ba8420"
server_key_version: "encryptionkey@0"
I20260925 09:25:54.075645 26360 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260925 09:25:54.079995 26375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:25:54.081640 26360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 09:25:54.081918 26360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data,/tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal
uuid: "d4c204c1ced94ce5b99a2ebafd4f7470"
format_stamp: "Formatted at 2026-09-25 09:25:54 on dist-test-slave-4wm5"
server_key: "70d24232f68f14e2822076576fb7baa9"
server_key_iv: "5461b55a52a3676d9ee57cb007ba8420"
server_key_version: "encryptionkey@0"
I20260925 09:25:54.082188 26360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:25:54.100333 26360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:25:54.105705 26360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:25:54.106142 26360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:25:54.128340 26360 rpc_server.cc:307] RPC server started. Bound to: 127.3.82.60:44871
I20260925 09:25:54.128360 26426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.82.60:44871 every 8 connection(s)
I20260925 09:25:54.129969 26360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/data/info.pb
I20260925 09:25:54.132843 3400 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0mddRs/build/asan/bin/kudu as pid 26360
I20260925 09:25:54.133080 3400 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0mddRs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790327973894689-3400-0/minicluster-data/master-2/wal/instance
I20260925 09:25:54.133591 3400 external_mini_cluster.cc:1428] Setting key 5af86818dca53ec8a80a5c7d459d9083
I20260925 09:25:54.134684 26427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:25:54.143204 26278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 4
I20260925 09:25:54.146111 26427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.62@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:25:54.163723 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.143934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53162)
0925 09:25:54.144174 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:54.144197 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:54.144315 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:54.145398 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:54.145404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:54.145992 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:54.146215 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.146230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.147920 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.147928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.148831 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.148844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.149643 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.149649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.149920 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.150736 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:54.150772 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:54.154475 (+ 3703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:54.159997 (+ 5522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.160002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.160008 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.160587 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.161148 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.161153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.161157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.161360 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.162575 (+ 1215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:54.162583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:54.162784 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 09:25:54.162951 (+ 167us) client_negotiation.cc:240] Negotiation successful
0925 09:25:54.163295 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328354, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:54.165799 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.144305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.3.82.60:44871)
0925 09:25:54.145008 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.145015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.145038 (+ 23us) server_negotiation.cc:407] Connection header received
0925 09:25:54.145090 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.145095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.145180 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.145296 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.146438 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.147762 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.149005 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.149475 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.149617 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.154752 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.154762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.154774 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.154821 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.159792 (+ 4971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.160761 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.160766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.160769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.161019 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.161539 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.161545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.161549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.162406 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.164148 (+ 1742us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.164266 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.164698 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":285,"server-negotiator.queue_time_us":540,"thread_start_us":329,"threads_started":1}
I20260925 09:25:54.165904 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.135801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.3.82.60:44871)
0925 09:25:54.136672 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.136685 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.136730 (+ 45us) server_negotiation.cc:407] Connection header received
0925 09:25:54.138582 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.138608 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.139172 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.139579 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.141799 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.143553 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.145564 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.146438 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.146668 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.154571 (+ 7903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.154587 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.154614 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.154667 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.159619 (+ 4952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.160706 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.160718 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.160729 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.161078 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.161619 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.161625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.161629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.162408 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.162778 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.163992 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.164494 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":639,"spinlock_wait_cycles":25856,"thread_start_us":402,"threads_started":1}
I20260925 09:25:54.175004 26278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: Bootstrap starting.
I20260925 09:25:54.177686 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.150225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.62:35687 (local address 127.0.0.1:48374)
0925 09:25:54.153603 (+ 3378us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:54.153670 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:54.153921 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:54.154963 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:54.154974 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:54.156234 (+ 1260us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:54.156758 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.156775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.158753 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.158762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.160747 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.160760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.162132 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.162140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.162335 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.163247 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:54.163270 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:54.167962 (+ 4692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:54.173755 (+ 5793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.173776 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.173785 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.174560 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.175088 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.175094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.175098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.175320 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.176178 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:54.176188 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:54.176432 (+ 244us) client_negotiation.cc:769] Sending connection context
0925 09:25:54.176668 (+ 236us) client_negotiation.cc:240] Negotiation successful
0925 09:25:54.177162 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2975,"thread_start_us":2760,"threads_started":1}
I20260925 09:25:54.177703 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.150261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.3.82.62:35687)
0925 09:25:54.154546 (+ 4285us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.154550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.154565 (+ 15us) server_negotiation.cc:407] Connection header received
0925 09:25:54.154646 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.154651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.154733 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.154849 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.157031 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.158524 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.160930 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.161797 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.162013 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.168339 (+ 6326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.168350 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.168361 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.168408 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.173539 (+ 5131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.174726 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.174733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.174737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.174914 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.175482 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.175490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.175497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.175983 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.176143 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.176838 (+ 695us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.177147 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4155,"thread_start_us":3967,"threads_started":1}
I20260925 09:25:54.179695 26278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:54.180415 26427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:25:54.181242 26278 log.cc:824] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: Log is configured to *not* fsync() on all Append() calls
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.62@KRBTEST.COM
I20260925 09:25:54.185639 26278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: No bootstrap required, opened a new log
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328354, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
I20260925 09:25:54.192626 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.170635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.3.82.62:35687)
0925 09:25:54.171486 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.171493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.171524 (+ 31us) server_negotiation.cc:407] Connection header received
0925 09:25:54.171779 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.171786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.171893 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.172030 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.173904 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.175180 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.177159 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.178065 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.178219 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.184791 (+ 6572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.184799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.184804 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.184850 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.189030 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.190129 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.190140 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.190144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.190350 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.190864 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.190870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.190874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.191267 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.191523 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.191897 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.192161 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":3214,"threads_started":1}
I20260925 09:25:54.198228 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.181296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.61:39857 (local address 127.0.0.1:35436)
0925 09:25:54.181576 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:54.181602 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:54.181880 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:54.182809 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:54.182817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:54.183342 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:54.183631 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.183645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.185194 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.185198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.186750 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.186767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.187800 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.187807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.188021 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.188652 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:54.188671 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:54.191143 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:54.195351 (+ 4208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.195357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.195361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.195811 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.196342 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.196349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.196352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.196530 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.197164 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:54.197169 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:54.197342 (+ 173us) client_negotiation.cc:769] Sending connection context
0925 09:25:54.197489 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 09:25:54.197805 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 09:25:54.198264 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.181416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.3.82.61:39857)
0925 09:25:54.181587 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.181592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.181723 (+ 131us) server_negotiation.cc:407] Connection header received
0925 09:25:54.182309 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.182316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.182416 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.182518 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.183824 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.184933 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.186992 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.187616 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.187781 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.191370 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.191377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.191382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.191427 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.195163 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.195959 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.195965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.195969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.196142 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.196712 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.196717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.196721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.197034 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.197206 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.197631 (+ 425us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.197880 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 09:25:54.197911 26278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.198453 26278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dfcb774ce98414696fcd1dcea831114, State: Initialized, Role: FOLLOWER
I20260925 09:25:54.199195 26278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.200423 26437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.200989 26437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.202361 26278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:54.203161 26427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.82.61@KRBTEST.COM
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328354, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.60@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:25:54.216673 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.195091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.3.82.61:39857)
0925 09:25:54.198519 (+ 3428us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.198524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.198542 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:25:54.198602 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.198606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.198677 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.198774 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.200807 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.201737 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.203473 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.204132 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.204260 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.209185 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.209191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.209195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.209234 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.212738 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.213718 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.213725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.213729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.213937 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.214554 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.214631 (+ 77us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.214635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.215014 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.215302 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.215975 (+ 673us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.216241 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3300,"thread_start_us":6927,"threads_started":1}
I20260925 09:25:54.222894 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.204102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53176)
0925 09:25:54.204493 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:54.204512 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:54.204637 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:54.205132 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:54.205138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:54.205700 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:54.205937 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.205950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.207800 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.207808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.209695 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.209713 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.210749 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.210757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.210972 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.213028 (+ 2056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:54.213053 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:54.215969 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:54.219954 (+ 3985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.219959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.219963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.220423 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.220899 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.220905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.220908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.221069 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.221787 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:54.221793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:54.221987 (+ 194us) client_negotiation.cc:769] Sending connection context
0925 09:25:54.222160 (+ 173us) client_negotiation.cc:240] Negotiation successful
0925 09:25:54.222515 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260925 09:25:54.224208 3400 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 09:25:54.224152 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.204334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.3.82.60:44871)
0925 09:25:54.204827 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.204832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.204850 (+ 18us) server_negotiation.cc:407] Connection header received
0925 09:25:54.204897 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.204903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.204989 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.205090 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.206125 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.207601 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.209955 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.210435 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.210604 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.216179 (+ 5575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.216186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.216191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.216227 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.219829 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.220556 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.220561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.220576 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.220773 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.221185 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.221189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.221192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.221598 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.221738 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.223600 (+ 1862us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.223791 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392}
I20260925 09:25:54.227386 26253 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9dfcb774ce98414696fcd1dcea831114"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48378
I20260925 09:25:54.227918 26253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 09:25:54.228168 26253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:25:54.230096 26232 tablet.cc:2369] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:25:54.230698 26427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: Bootstrap starting.
I20260925 09:25:54.220328 26211 master_runner.cc:186] Error getting master registration for 127.3.82.61:39857: OK, instance_id { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" instance_seqno: 1790328353556475 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:25:54.233495 26253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.235096 26427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: Neither blocks nor log segments found. Creating new log.
W20260925 09:25:54.236446 26448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:25:54.236599 26448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:25:54.236559 26427 log.cc:824] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:54.236948 26211 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:25:54.237468 26253 leader_election.cc:288] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [CANDIDATE]: Term 1 election: Requested vote from peers 4dabf75a321b4816b44958a67a8feeac (127.3.82.61:39857), d4c204c1ced94ce5b99a2ebafd4f7470 (127.3.82.60:44871)
I20260925 09:25:54.240245 26427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: No bootstrap required, opened a new log
I20260925 09:25:54.252157 26427 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.252689 26427 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c204c1ced94ce5b99a2ebafd4f7470, State: Initialized, Role: FOLLOWER
I20260925 09:25:54.253265 26427 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.254980 26455 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.256387 26427 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:54.256573 26402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9dfcb774ce98414696fcd1dcea831114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470"
I20260925 09:25:54.257113 26402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:54.261759 26402 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dfcb774ce98414696fcd1dcea831114 in term 1.
I20260925 09:25:54.264559 26228 leader_election.cc:302] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dfcb774ce98414696fcd1dcea831114, d4c204c1ced94ce5b99a2ebafd4f7470; no voters:
I20260925 09:25:54.265420 26437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:25:54.276890 26455 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.279531 26437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [term 1 LEADER]: Becoming Leader. State: Replica: 9dfcb774ce98414696fcd1dcea831114, State: Running, Role: LEADER
I20260925 09:25:54.280457 26437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.292074 26352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } attempt: 5
I20260925 09:25:54.303884 26460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dfcb774ce98414696fcd1dcea831114. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.304483 26460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: This master's current role is: LEADER
Sep 25 09:25:54 dist-test-slave-4wm5 krb5kdc[26198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790328353, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.82.61@KRBTEST.COM for kudu/127.3.82.60@KRBTEST.COM
I20260925 09:25:54.312088 26464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:25:54.313027 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.263308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53184 (local address 127.3.82.60:44871)
0925 09:25:54.263549 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.263562 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.276895 (+ 13333us) server_negotiation.cc:407] Connection header received
0925 09:25:54.282443 (+ 5548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.282462 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.282569 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.282662 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.287583 (+ 4921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.288857 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.293217 (+ 4360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.294121 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.294263 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.301643 (+ 7380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.301651 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.301659 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.301704 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.305965 (+ 4261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.308091 (+ 2126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.308100 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.308107 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.308337 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.309308 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.309321 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.309327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.309886 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.310190 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.311406 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.311996 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":89}
I20260925 09:25:54.314234 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.292786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.82.60:44871 (local address 127.0.0.1:53194)
0925 09:25:54.293093 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0925 09:25:54.293117 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 09:25:54.293247 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:25:54.293806 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 09:25:54.293812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 09:25:54.294482 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0925 09:25:54.294769 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.294785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.296963 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.296977 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.298831 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 09:25:54.298853 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.300359 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.300366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 09:25:54.300656 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.301440 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 09:25:54.301475 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 09:25:54.305544 (+ 4069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 09:25:54.310621 (+ 5077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.310628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.310632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.311338 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.311851 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 09:25:54.311859 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 09:25:54.311864 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 09:25:54.312088 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 09:25:54.312959 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 09:25:54.312966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 09:25:54.313197 (+ 231us) client_negotiation.cc:769] Sending connection context
0925 09:25:54.313358 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 09:25:54.313753 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260925 09:25:54.315451 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.292870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.3.82.60:44871)
0925 09:25:54.293093 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.293101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.293362 (+ 261us) server_negotiation.cc:407] Connection header received
0925 09:25:54.293418 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.293423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.293506 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.293600 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.294963 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.296749 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.299072 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.299921 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.300101 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.305880 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.305889 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.305894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.305948 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.310390 (+ 4442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.311504 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.311510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.311514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.311661 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.312267 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.312273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.312277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.312711 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.312879 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.314744 (+ 1865us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.315044 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 09:25:54.323860 26352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: Bootstrap starting.
I20260925 09:25:54.324503 26464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:25:54.331888 26352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: Neither blocks nor log segments found. Creating new log.
I20260925 09:25:54.334784 26352 log.cc:824] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: Log is configured to *not* fsync() on all Append() calls
I20260925 09:25:54.336498 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.258864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.3.82.62:35687)
0925 09:25:54.262406 (+ 3542us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.262415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.262445 (+ 30us) server_negotiation.cc:407] Connection header received
0925 09:25:54.262542 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.262548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.262706 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.262869 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.284173 (+ 21304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.285568 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.289720 (+ 4152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.290779 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.290915 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.326067 (+ 35152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.326076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.326081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.326135 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.331226 (+ 5091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.332423 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.332430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.332434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.332646 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.333231 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.333236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.333239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.333720 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.334187 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.335621 (+ 1434us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.335925 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3300}
I20260925 09:25:54.349807 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:54.260000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.3.82.61:39857)
0925 09:25:54.260196 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.260205 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.278736 (+ 18531us) server_negotiation.cc:407] Connection header received
0925 09:25:54.286649 (+ 7913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.286659 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.286780 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.286871 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.296833 (+ 9962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.298555 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.318722 (+ 20167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:54.319875 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:54.320069 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:54.339109 (+ 19040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 09:25:54.339121 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 09:25:54.339129 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 09:25:54.339197 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0925 09:25:54.344588 (+ 5391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.346101 (+ 1513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.346111 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.346118 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.346366 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 09:25:54.347024 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 09:25:54.347032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 09:25:54.347039 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 09:25:54.347576 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 09:25:54.347949 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:54.348786 (+ 837us) server_negotiation.cc:299] Negotiation successful
0925 09:25:54.349228 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260925 09:25:54.358760 26381 tablet.cc:2369] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:25:54.370625 26474 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:25:54.370887 26474 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:25:54.383924 26402 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [term 1 FOLLOWER]: Refusing update from remote peer 9dfcb774ce98414696fcd1dcea831114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:54.386025 26352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: No bootstrap required, opened a new log
I20260925 09:25:54.387573 26460 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:54.376924 26360 master_runner.cc:186] Error getting master registration for 127.3.82.61:39857: OK, instance_id { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" instance_seqno: 1790328353556475 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:25:54.392110 26360 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:25:54.399029 26352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.399667 26352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dabf75a321b4816b44958a67a8feeac, State: Initialized, Role: FOLLOWER
I20260925 09:25:54.400405 26352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } }
I20260925 09:25:54.406991 26476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.407511 26476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.409749 26455 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dfcb774ce98414696fcd1dcea831114. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.410202 26455 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.410789 26352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:25:54.411285 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9dfcb774ce98414696fcd1dcea831114" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4dabf75a321b4816b44958a67a8feeac"
I20260925 09:25:54.412936 26326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:25:54.417085 26326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [term 1 FOLLOWER]: Refusing update from remote peer 9dfcb774ce98414696fcd1dcea831114: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:25:54.423130 26466 mvcc.cc:205] Tried to move back new op lower bound from 7333184939470856192 to 7333184939141087232. Current Snapshot: MvccSnapshot[applied={T|T < 7333184939470856192}]
I20260925 09:25:54.420993 26327 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dfcb774ce98414696fcd1dcea831114 in term 1.
I20260925 09:25:54.431413 26460 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:25:54.437219 26477 mvcc.cc:205] Tried to move back new op lower bound from 7333184939470856192 to 7333184939141087232. Current Snapshot: MvccSnapshot[applied={T|T < 7333184939470856192}]
I20260925 09:25:54.446244 26455 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.446887 26455 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.457736 26476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dfcb774ce98414696fcd1dcea831114. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.458256 26476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.458840 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.459474 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:25:54.470255 26495 catalog_manager.cc:1259] Loaded cluster ID: 1ee7781ef3bf4ee5b5221c33362c32de
I20260925 09:25:54.470405 26495 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: loading cluster ID for follower catalog manager: success
W20260925 09:25:54.471975 26495 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260925 09:25:54.472162 26495 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:25:54.472955 26437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.473366 26437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:54.473598 26437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9dfcb774ce98414696fcd1dcea831114" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfcb774ce98414696fcd1dcea831114" member_type: VOTER last_known_addr { host: "127.3.82.62" port: 35687 } } peers { permanent_uuid: "4dabf75a321b4816b44958a67a8feeac" member_type: VOTER last_known_addr { host: "127.3.82.61" port: 39857 } } peers { permanent_uuid: "d4c204c1ced94ce5b99a2ebafd4f7470" member_type: VOTER last_known_addr { host: "127.3.82.60" port: 44871 } } }
I20260925 09:25:54.473858 26437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114 [sys.catalog]: This master's current role is: LEADER
I20260925 09:25:54.481480 26464 catalog_manager.cc:1347] Generated new cluster ID: 1ee7781ef3bf4ee5b5221c33362c32de
I20260925 09:25:54.481628 26464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:25:54.518193 26464 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:25:54.520592 26464 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:25:54.566254 26464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dfcb774ce98414696fcd1dcea831114: Generated new TSK 0
I20260925 09:25:54.567647 26464 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:25:54.732637 26434 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.722597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.3.82.62:35687)
0925 09:25:54.722847 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.722857 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.730468 (+ 7611us) server_negotiation.cc:407] Connection header received
0925 09:25:54.730826 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.730834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.730940 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.731022 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.732350 (+ 1328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48396: BlockingRecv error: recv got EOF from 127.0.0.1:48396 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 09:25:54.735343 26431 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.729153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.3.82.60:44871)
0925 09:25:54.729398 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.729407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.733499 (+ 4092us) server_negotiation.cc:407] Connection header received
0925 09:25:54.733780 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.733787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.733929 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.734009 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.735059 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53198: BlockingRecv error: recv got EOF from 127.0.0.1:53198 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260925 09:25:54.740644 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.732586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.3.82.61:39857)
0925 09:25:54.732845 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.732853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.736288 (+ 3435us) server_negotiation.cc:407] Connection header received
0925 09:25:54.738795 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.738802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.738908 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.738987 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.740211 (+ 1224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35466: BlockingRecv error: recv got EOF from 127.0.0.1:35466 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 09:25:54.788753 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.777695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.3.82.61:39857)
0925 09:25:54.777915 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.777922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.786510 (+ 8588us) server_negotiation.cc:407] Connection header received
0925 09:25:54.786927 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.786936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.787097 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.787232 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.788429 (+ 1197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35468: BlockingRecv error: recv got EOF from 127.0.0.1:35468 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260925 09:25:54.840545 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.825006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.3.82.61:39857)
0925 09:25:54.825316 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.825325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.838534 (+ 13209us) server_negotiation.cc:407] Connection header received
0925 09:25:54.838806 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.838812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.838919 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.839030 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.840259 (+ 1229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35484: BlockingRecv error: recv got EOF from 127.0.0.1:35484 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
W20260925 09:25:54.905246 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.890574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.3.82.61:39857)
0925 09:25:54.890776 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.890782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.903162 (+ 12380us) server_negotiation.cc:407] Connection header received
0925 09:25:54.903546 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.903553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.903699 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.903789 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.904947 (+ 1158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35500: BlockingRecv error: recv got EOF from 127.0.0.1:35500 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 09:25:54.966815 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.953283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35514 (local address 127.3.82.61:39857)
0925 09:25:54.953540 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.953547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.964441 (+ 10894us) server_negotiation.cc:407] Connection header received
0925 09:25:54.965062 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.965070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.965191 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.965278 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.966511 (+ 1233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35514: BlockingRecv error: recv got EOF from 127.0.0.1:35514 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
W20260925 09:25:54.986753 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:54.984191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35516 (local address 127.3.82.61:39857)
0925 09:25:54.984394 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:54.984401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:54.984986 (+ 585us) server_negotiation.cc:407] Connection header received
0925 09:25:54.985239 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:54.985245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:54.985346 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:54.985462 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:54.986467 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35516: BlockingRecv error: recv got EOF from 127.0.0.1:35516 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260925 09:25:55.033761 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:55.031331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.3.82.61:39857)
0925 09:25:55.031565 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.031573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.032116 (+ 543us) server_negotiation.cc:407] Connection header received
0925 09:25:55.032342 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.032350 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.032463 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.032598 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:55.033481 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35528: BlockingRecv error: recv got EOF from 127.0.0.1:35528 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260925 09:25:55.072070 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:55.069954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.3.82.61:39857)
0925 09:25:55.070189 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.070195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.070488 (+ 293us) server_negotiation.cc:407] Connection header received
0925 09:25:55.070723 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.070728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.070818 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.071005 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:55.071790 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35542: BlockingRecv error: recv got EOF from 127.0.0.1:35542 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260925 09:25:55.142675 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:55.140676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.3.82.61:39857)
0925 09:25:55.140860 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.140867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.140966 (+ 99us) server_negotiation.cc:407] Connection header received
0925 09:25:55.141216 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.141222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.141314 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.141427 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:55.142450 (+ 1023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35546: BlockingRecv error: recv got EOF from 127.0.0.1:35546 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260925 09:25:55.281998 26357 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:25:55.279661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.3.82.61:39857)
0925 09:25:55.279955 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.279966 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.280480 (+ 514us) server_negotiation.cc:407] Connection header received
0925 09:25:55.280643 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.280648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.280730 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.280831 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 09:25:55.281697 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35558: BlockingRecv error: recv got EOF from 127.0.0.1:35558 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260925 09:25:55.373178 26474 catalog_manager.cc:1259] Loaded cluster ID: 1ee7781ef3bf4ee5b5221c33362c32de
I20260925 09:25:55.373279 26474 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: loading cluster ID for follower catalog manager: success
I20260925 09:25:55.375996 26474 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: acquiring CA information for follower catalog manager: success
I20260925 09:25:55.378046 26474 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4c204c1ced94ce5b99a2ebafd4f7470: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:25:55.475797 26495 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: acquiring CA information for follower catalog manager: success
I20260925 09:25:55.477965 26495 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4dabf75a321b4816b44958a67a8feeac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:25:55.551878 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:55.543954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.3.82.61:39857)
0925 09:25:55.544250 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.544257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.544646 (+ 389us) server_negotiation.cc:407] Connection header received
0925 09:25:55.544836 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.544840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.545028 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.545139 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:55.545594 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.546619 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.548893 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.549444 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.549580 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:55.550412 (+ 832us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:55.551125 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:55.551273 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:55.551435 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 09:25:55.551589 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260925 09:25:55.568449 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:55.559308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.3.82.60:44871)
0925 09:25:55.559850 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.559855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.559875 (+ 20us) server_negotiation.cc:407] Connection header received
0925 09:25:55.560191 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.560195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.560360 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.560536 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:55.561030 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.561795 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.563966 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.564735 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.564909 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:55.566138 (+ 1229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:55.566991 (+ 853us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:55.567195 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:55.567600 (+ 405us) server_negotiation.cc:299] Negotiation successful
0925 09:25:55.567755 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":304,"threads_started":1}
I20260925 09:25:55.589115 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:55.576360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.3.82.62:35687)
0925 09:25:55.581407 (+ 5047us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.581419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.582215 (+ 796us) server_negotiation.cc:407] Connection header received
0925 09:25:55.582381 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.582388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.582695 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.582923 (+ 228us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:55.583628 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.584488 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.586788 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.587272 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.587434 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:55.587968 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:55.588337 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:55.588487 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:55.588665 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 09:25:55.588766 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4848,"thread_start_us":4712,"threads_started":1}
I20260925 09:25:55.590864 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:55.582192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.3.82.61:39857)
0925 09:25:55.582474 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.582484 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.583720 (+ 1236us) server_negotiation.cc:407] Connection header received
0925 09:25:55.583892 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.583897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.584010 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.584108 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:55.584981 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.586161 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.588452 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.589085 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.589245 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:55.589972 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:55.590142 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:55.590217 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:55.590458 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 09:25:55.590572 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 09:25:55.591300 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 09:25:55.576716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.3.82.60:44871)
0925 09:25:55.576954 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 09:25:55.576961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 09:25:55.583370 (+ 6409us) server_negotiation.cc:407] Connection header received
0925 09:25:55.583583 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 09:25:55.583588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 09:25:55.583708 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 09:25:55.583807 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 09:25:55.585025 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.585919 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.588444 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 09:25:55.589604 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 09:25:55.589712 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 09:25:55.590365 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 09:25:55.590509 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 09:25:55.590572 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 09:25:55.590783 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 09:25:55.590871 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 09:25:55.596061 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 26211
I20260925 09:25:55.597255 26273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:55.725540 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 26211
I20260925 09:25:55.747097 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 26285
I20260925 09:25:55.748394 26347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:55.861413 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 26285
I20260925 09:25:55.882360 3400 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0mddRs/build/asan/bin/kudu pid 26360
I20260925 09:25:55.883636 26422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 09:25:56.002246 3400 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0mddRs/build/asan/bin/kudu with pid 26360
2026-09-25T09:25:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4920 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10054 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (382066 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST